Token

A token has properties: text, type, line, character position in the line (so we can ignore tabs), token channel, index, and source from which we obtained this token.

Members

Functions

getChannel
int getChannel()

Return the channel this token. Each token can arrive at the parser on a different channel, but the parser only "tunes" to a single channel. The parser ignores everything not on DEFAULT_CHANNEL.

getCharPositionInLine
int getCharPositionInLine()

The index of the first character of this token relative to the beginning of the line at which it occurs, 0..n-1

getInputStream
CharStream getInputStream()

Gets the {@link CharStream} from which this token was derived.

getLine
int getLine()

The line number on which the 1st character of this token was matched, line=1..n

getStartIndex
int getStartIndex()

The starting character index of the token This method is optional; return -1 if not implemented.

getStopIndex
int getStopIndex()

The last character index of the token. This method is optional; return -1 if not implemented.

getText
string getText()

Get the text of the token.

getTokenIndex
int getTokenIndex()

An index from 0..n-1 of the token object in the input stream. This must be valid in order to print token streams and use TokenRewriteStream.

getTokenSource
TokenSource getTokenSource()

Gets the {@link TokenSource} which created this token.

getType
int getType()

Get the token type of the token

Meta