Token

A token has properties: text, type, line, character position in the line (so we can ignore tabs), token channel, index, and source from which we obtained this token.

interface Token {

string getText();

int getType();

int getLine();

int getCharPositionInLine();

int getChannel();

int getTokenIndex();

int getStartIndex();

int getStopIndex();

TokenSource getTokenSource();

CharStream getInputStream();

}

Members

Functions

getChannel int getChannel(): Return the channel this token. Each token can arrive at the parser on a different channel, but the parser only "tunes" to a single channel. The parser ignores everything not on DEFAULT_CHANNEL.
getCharPositionInLine int getCharPositionInLine(): The index of the first character of this token relative to the beginning of the line at which it occurs, 0..n-1
getInputStream CharStream getInputStream(): Gets the {@link CharStream} from which this token was derived.
getLine int getLine(): The line number on which the 1st character of this token was matched, line=1..n
getStartIndex int getStartIndex(): The starting character index of the token This method is optional; return -1 if not implemented.
getStopIndex int getStopIndex(): The last character index of the token. This method is optional; return -1 if not implemented.
getText string getText(): Get the text of the token.
getTokenIndex int getTokenIndex(): An index from 0..n-1 of the token object in the input stream. This must be valid in order to print token streams and use TokenRewriteStream.
getTokenSource TokenSource getTokenSource(): Gets the {@link TokenSource} which created this token.
getType int getType(): Get the token type of the token

Token

Members

Functions

Meta

Source