Crate lrlex

source ·
Expand description

lrlex is a partial replacement for lex / flex. It takes in a .l file and statically compiles it to Rust code. The resulting LRNonStreamingLexerDef can then be given an input string, from which it instantiates an LRNonStreamingLexer. This provides an iterator which can produce the sequence of lrpar::Lexemes for that input, as well as answer basic queries about cfgrammar::Spans (e.g. extracting substrings, calculating line and column numbers).


A convenience macro for including statically compiled .l files. A file src/a/b/c.l processed by CTLexerBuilder::lexer_in_src_dir can then be used in a crate with lrlex_mod!("a/b/c.l").


An interface to the result of CTLexerBuilder::build().
A CTLexerBuilder allows one to specify the criteria for building a statically generated lexer.
lrlex’s standard lexeme struct: all lexemes are instances of this struct.
A Lexing error.
An LRNonStreamingLexer holds a reference to a string and can lex it into lrpar::Lexemes. Although the struct is tied to a single string, no guarantees are made about whether the lexemes are cached or not.
This struct represents, in essence, a .l file in memory. From it one can produce an LRNonStreamingLexer which actually lexes inputs.
Any error from the Lex parser returns an instance of this struct.


The various different possible Lex parser errors.
Specifies the Rust Edition that will be emitted during code generation.
Specify the visibility of the module generated by CTLexerBuilder.


Methods which all lexer definitions must implement.


Create a Rust module named mod_name that can be imported with lrlex_mod!(mod_name). The module contains one const StorageT per token in token_map, with the token prefixed by T_. For example with StorageT u8, mod_name x, and token_map HashMap{"ID": 0, "INT": 1} the generated module will look roughly as follows:

Type Definitions