Constructs a new {@link CommonTokenStream} using the specified token source and the default token channel ({@link Token#DEFAULT_CHANNEL}).
Constructs a new {@link CommonTokenStream} using the specified token source and filtering tokens to the specified channel. Only tokens whose {@link Token#getChannel} matches {@code channel} or have the {@link Token#getType} equal to {@link Token#EOF} will be returned by the token stream lookahead methods.
@uml @override
@uml @override
@uml @override
Count EOF just once.
Specifies the channel to use for filtering tokens.
The {@link TokenSource} from which tokens for this stream are fetched.
A collection of all tokens fetched from the token source. The list is considered a complete view of the input once {@link #fetchedEOF} is set to {@code true}.
Indicates whether the {@link Token#EOF} token has been fetched from {@link #tokenSource} and added to {@link #tokens}. This field improves performance for the following cases:
@uml @override
Make sure index {@code i} in tokens has a token.
Add {@code n} elements to buffer.
@uml @override
Get all tokens from start..stop inclusively
@uml @override
Allowed derived classes to modify the behavior of operations which change the current stream position by adjusting the target token index of a seek operation. The default implementation simply returns {@code i}. If an exception is thrown in this method, the current stream index should not be changed.
Reset this token stream by setting its token source.
Given a start and stop index, return a List of all tokens in the token type BitSet. Return null if no tokens were found. This method looks at both on and off channel tokens.
Given a starting index, return the index of the next token on channel. Return {@code i} if {@code tokensi} is on channel. Return the index of the EOF token if there are no tokens on channel between {@code i} and EOF.
Given a starting index, return the index of the previous token on channel. Return {@code i} if {@code tokensi} is on channel. Return -1 if there are no tokens on channel between {@code i} and 0.
Collect all tokens on specified channel to the right of the current token up until we see a token on DEFAULT_TOKEN_CHANNEL or EOF. If channel is -1, find any non default channel token.
Collect all hidden tokens (any off-default channel) to the right of the current token up until we see a token on DEFAULT_TOKEN_CHANNEL of EOF.
Collect all tokens on specified channel to the left of the current token up until we see a token on DEFAULT_TOKEN_CHANNEL. If channel is -1, find any non default channel token.
Collect all hidden tokens (any off-default channel) to the left of the current token up until we see a token on DEFAULT_TOKEN_CHANNEL.
Collect all hidden tokens (any off-default channel) to the left of the current token up until we see a token on DEFAULT_TOKEN_CHANNEL.
@uml @override
@uml @override
@uml @override
@uml @override
Get all tokens from lexer until EOF
This class extends {@link BufferedTokenStream} with functionality to filter token streams to tokens on a particular channel (tokens where {@link Token#getChannel} returns a particular value). <p> This token stream provides access to all tokens by index or when calling methods like {@link #getText}. The channel filtering is only used for code accessing tokens via the lookahead methods {@link #LA}, {@link #LT}, and {@link #LB}.</p>
<p> By default, tokens are placed on the default channel ({@link Token#DEFAULT_CHANNEL}), but may be reassigned by using the {@code ->channel(HIDDEN)} lexer command, or by using an embedded action to call {@link Lexer#setChannel}. </p>
<p> Note: lexer rules which use the {@code ->skip} lexer command or call {@link Lexer#skip} do not produce tokens at all, so input text matched by such a rule will not be available as part of the token stream, regardless of channel.</p>