Starts tokenizing the buffer for coloring and parenthesis matching.
The token-sym->style argument will be passed the first return symbol from get-token
and should return the style-name that the token should be colored.
The get-token argument takes an input port and returns the next token as 5 values:
An unused value. This value is intended to represent the textual
component of the token and may be used as such in the future.
A symbol describing the type of the token. This symbol is transformed
into a style-name via the token-sym->style argument. The symbols
'white-space and 'comment have special meaning and should always be
returned for white space and comment tokens respectively. The symbol
'no-color can be used to indicate that although the token is not white
space, it should not be colored. The symbol 'eof must be used to
indicate when all the tokens have been consumed.
A symbol indicating how the token should be treated by the paren
matcher or #f. This symbol should be in the pairs argument.
The starting position of the token.
The ending position of the token.
The get-token function will usually be implemented with a lexer using the
parser-tools/lex library.
get-token must obey the following invariants:
Every position in the buffer must be accounted for in exactly one
token.
The token returned by get-token must rely only on the contents of the
input port argument. This means that the tokenization of some part of
the input cannot depend on earlier parts of the input.
No edit to the buffer can change the tokenization of the buffer prior
to the token immediately preceding the edit. In the following
example this invariant does not hold. If the buffer contains:
and the tokenizer treats the unmatched " as its own token (a string
error token), and separately tokenizes the 1 2 and 3, an edit to make
the buffer look like:
would result in a single string token modifying previous tokens. To
handle these situations, get-token must treat the first line as a
single token.
The pairs argument is a list of different kinds of matching parens. The second
value returned by get-token is compared to this list to see how the
paren matcher should treat the token. An example: Suppose pairs is
'((|(| |)|) (|[| |]|) (begin end)). This means that there
are three kinds of parens. Any token which has 'begin as its second
return value will act as an open for matching tokens with 'end.
Similarly any token with '|]| will act as a closing match for
tokens with '|[|. When trying to correct a mismatched
closing parenthesis, each closing symbol in pairs will be converted to
a string and tried as a closing parenthesis.
Stops coloring and paren matching the buffer.
If clear-colors is true all the text in the buffer will have its
style set to Standard.
Causes the entire tokenizing/coloring system to become inactive.
Intended for debugging purposes only.
stop? determines whether the system is being forced to stop or allowed
to wake back up.
Indicates if the colorer for this editor has been stopped, or not.
Keep the text tokenized and paren matched, but stop altering the colors.
freeze-colorer will not return until the coloring/tokenization of the
entire text is brought up-to-date. It must not be called on a locked
text.
|
| recolor : boolean = #t |
| retokenize : boolean = #f |
Start coloring a frozen buffer again.
If recolor? is #t, the text is re-colored. If it is
#f the text is not recolored. When recolor? is #t,
retokenize? controls how the text is recolored. #f causes
the text to be entirely re-colored before thaw-colorer returns using
the existing tokenization. #t causes the entire text to be
retokenized and recolored from scratch. This will happen in the
background after the call to thaw-colorer returns.
| (send a-color:text reset-region start end) → void |
| start : natural-number? |
| end : (union 'end natural-number?) |
Set the region of the text that is tokenized.
Sets the currently active regions to be regions.
This returns the list of regions that are currently being colored in the editor.
|
| position : natural-number? |
| direction : (symbols 'forward 'backward) |
| comments? : boolean? |
Returns the next non-whitespace character.
Starts from position and skips whitespace in the direction indicated
by direction. If comments? is true, comments are skipped as well as
whitespace. skip-whitespace determines whitespaces and comments by
comparing the token type to 'white-space and 'comment.
Must only be called while the tokenizer is started.
|
| → (union natural-number? false?) |
| position : natural-number? |
| cutoff : natural-number? |
Skip all consecutive whitespaces and comments (using skip-whitespace)
immediately preceding the position. If the token at this position is
a close, return the position of the matching open, or #f if
there is none. If the token was an open, return #f. For any
other token, return the start of that token.
Must only be called while the tokenizer is started.
|
| → (union natural-number? false?) |
| position : natural-number? |
| cutoff : natural-number? |
Return the starting position of the interior of the (non-atomic)
s-expression containing position, or #f is there is none.
Must only be called while the tokenizer is started.
|
| → (union natural-number? false?) |
| position : natural-number? |
| cutoff : natural-number? |
Skip all consecutive whitespaces and comments (using skip-whitespace)
immediately following position. If the token at this position is an
open, return the position of the matching close, or #f if
there is none. For any other token, return the end of that token.
Must only be called while the tokenizer is started.
Position is the place to put the parenthesis and char is the
parenthesis to be added. If fixup? is true, the right kind of closing
parenthesis will be chosen from the pairs list kept last passed to
start-colorer, otherwise char will be inserted, even if it is not the
right kind. If flash? is true the matching open parenthesis will be
flashed.
Return a symbol for the lexer-determined token type for the token that
contains the item after position.
Must only be called while the tokenizer is started.