CommonTokenFactory

This default implementation of {@link TokenFactory} creates {@link CommonToken} objects. <p> This token factory does not explicitly copy token text when constructing tokens.</p>

class CommonTokenFactory : TokenFactory!CommonToken {

protected

bool copyText;

this(bool copyText);

this();

CommonToken create(TokenFactorySourcePair source, int type, Variant text, int channel, size_t start, size_t stop, int line, int charPositionInLine);

CommonToken create(int type, Variant text);

static CommonTokenFactory DEFAULT();

}

Constructors

this this(bool copyText): @uml Constructs a {@link CommonTokenFactory} with the specified value for {@link #copyText}.
this this(): @uml Constructs a {@link CommonTokenFactory} with {@link #copyText} set to {@code false}.

Members

Functions

create CommonToken create(TokenFactorySourcePair source, int type, Variant text, int channel, size_t start, size_t stop, int line, int charPositionInLine): Undocumented in source. Be warned that the author may not have intended to support it.
create CommonToken create(int type, Variant text): Undocumented in source. Be warned that the author may not have intended to support it.

Static functions

DEFAULT CommonTokenFactory DEFAULT()

Variables

copyText bool copyText;: @uml Indicates whether {@link CommonToken#setText} should be called after constructing tokens to explicitly set the text. This is useful for cases where the input stream might not be able to provide arbitrary substrings of text from the input after the lexer creates a token (e.g. the implementation of {@link CharStream#getText} in {@link UnbufferedCharStream} throws an {@link UnsupportedOperationException}). Explicitly setting the token text allows {@link Token#getText} to be called at any time regardless of the input stream implementation.