Trait lrpar::Lexeme

pub trait Lexeme<StorageT>: Debug + Display + Eq + Hash + Copy {
    // Required methods
    fn new(tok_id: StorageT, start: usize, len: usize) -> Self
       where Self: Sized;
    fn new_faulty(tok_id: StorageT, start: usize, len: usize) -> Self
       where Self: Sized;
    fn tok_id(&self) -> StorageT;
    fn span(&self) -> Span;
    fn faulty(&self) -> bool;
}

A lexeme represents a segment of the user’s input that conforms to a known type: this trait captures the common behaviour of all lexeme structs.

Lexemes are assumed to have a definition which describes all possible correct lexemes (e.g. the regular expression [0-9]+ defines all integer lexemes). This trait also allows “faulty” lexemes to be represented – that is, lexemes that have resulted from error recovery of some sort. Faulty lexemes can violate the lexeme’s type definition in any possible way (e.g. they might span more or less input than the definition would suggest is possible).

Required Methods

fn new(tok_id: StorageT, start: usize, len: usize) -> Self

where Self: Sized,

Create a new lexeme with ID tok_id, a starting position in the input start, and length len.

Lexemes created using this function are expected to be “correct” in the sense that they fully respect the lexeme’s definition semantics. To create faulty lexemes, use new_faulty.

fn new_faulty(tok_id: StorageT, start: usize, len: usize) -> Self

where Self: Sized,

Create a new faulty lexeme with ID tok_id and a starting position in the input start.

fn tok_id(&self) -> StorageT

The token ID.

fn span(&self) -> Span

Obtain this Lexeme’s Span.

fn faulty(&self) -> bool

Returns true if this lexeme is “faulty” i.e. is the result of error recovery in some way. If true, note that the lexeme’s span may be greater or less than you may expect from the lexeme’s definition.

Object Safety

This trait is not object safe.

Implementors