Lex extensions

Flags can be specified at compile time through LexFlags or at .l file parse time using a %grmtools{ } section. At compile time these flags can be enabled using CTLexerBuilder methods.

Flags commonly affect the parsing of the lex file, the interpretation regular expressions, and set limits.

Boolean flags are specified by their name, and can be negated by prefixing with ! other flags should specify their value immediately after the flag name.

Example

%grmtools {
    allow_wholeline_comments,
    !octal,
    size_limit: 1024,
}
%%
. "rule"

List of flags:

FlagValueRequiredRegex1
posix_escapes2bool
allow_wholeline_comment3bool
case_insensitivebool
dot_matches_new_linebool
multi_linebool
octalbool
swap_greedbool
ignore_whitespacebool
unicodebool
size_limitusize
dfa_size_limitusize
nest_limitu32
2

Enable compatibility with posix escape sequences.

3

Enables rust style // comments at the start of lines. Which requires escaping of / when used in a regex.

1

✓ Flag gets passed directly to regex::RegexBuilder.

Flags affecting Posix compatibility

As discussed in Lex compatibility the default behaviors of grmtools and rust's regex library have differed from that of posix lex.

The following flags can change the behavior to match posix lex more closely.

%grmtools {
    !dot_matches_new_line,
    posix_escapes
}
%%
...