Struct CTTokenMapBuilder

pub struct CTTokenMapBuilder<StorageT: Display + ToTokens> { /* private fields */ }

Exports all token IDs used by a parser as a separate Rust module.

This builder will create a Rust module named mod_name that can be imported with lrlex_mod!(mod_name). The module will contain one const StorageT per token in token_map, with the token prefixed by T_. In addition, it will contain an array of all token IDs TOK_IDS.

For example, if StorageT is u8, mod_name is x, and token_map is HashMap{"ID": 0, "INT": 1} the generated module will look roughly as follows:

mod x {
  pub const T_ID: u8 = 0;
  pub const T_INT: u8 = 1;
  pub const TOK_IDS: &[u8] = &[T_ID, T_INT];
}

See the custom lexer example for more usage details.

Implementations

impl<StorageT: Display + ToTokens> CTTokenMapBuilder<StorageT>

pub fn new( mod_name: impl Into<String>, token_map: impl Borrow<HashMap<String, StorageT>>, ) -> Self

Create a new token map builder.

See the builder documentation for more info.

pub fn rename_map<M, I, K, V>(self, rename_map: Option<M>) -> Self

where M: IntoIterator<Item = I>, I: Borrow<(K, V)>, K: AsRef<str>, V: AsRef<str>,

Set a token rename map.

Rename map is used to specify identifier names for tokens whose names are not valid Rust identifiers. For example, if token_map is HashMap{"+": 0, "ID": 1} and rename_map is HashMap{"+": "PLUS"} then the generated module will look roughly as follows:

mod x {
  pub const T_PLUS: u8 = 0;
  pub const T_ID: u8 = 1;
}

pub fn allow_dead_code(self, allow_dead_code: bool) -> Self

Control whether the builder will add #[allow(dead_code)] to the generated module.

By default, all tokens are #[deny(dead_code)], meaning that you’ll get a warning if your custom lexer doesn’t use any of them. This function can be used to disable this behavior.

pub fn build(&self) -> Result<(), Box<dyn Error>>

Build the token map module.

Trait Implementations

impl<StorageT: Clone + Display + ToTokens> Clone for CTTokenMapBuilder<StorageT>

fn clone(&self) -> CTTokenMapBuilder<StorageT>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<StorageT: Debug + Display + ToTokens> Debug for CTTokenMapBuilder<StorageT>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.