de.dante.extex.scanner.stream.impl
Class TokenStreamBaseImpl

java.lang.Object
  extended byde.dante.extex.scanner.stream.impl.TokenStreamBaseImpl
All Implemented Interfaces:
TokenStream
Direct Known Subclasses:
TokenStreamImpl

public class TokenStreamBaseImpl
extends java.lang.Object
implements TokenStream

This is the base implementation of a token stream. It has an internal stack of tokens which can be enlarged with push() or reduced with pop().

Version:
$Revision: 1.28 $
Author:
Gerd Neugebauer, Michael Niedermair

Constructor Summary
TokenStreamBaseImpl(boolean isFile)
          Creates a new object.
TokenStreamBaseImpl(boolean isFile, Tokens tokens)
          Creates a new object.
 
Method Summary
 boolean closeFileStream()
          Close this stream if it is a file stream.
 Token get(TokenFactory factory, Tokenizer tokenizer)
          Get the next token from the token stream.
 Locator getLocator()
          Getter for the locator.
protected  Token getNext(TokenFactory factory, Tokenizer tokenizer)
          Get the next token when the stack is empty.
 boolean isEof()
          Test for end of file.
 boolean isEol()
          Check to see if the token stream is currently at the end of line.
 boolean isFileStream()
          Check whether the current stream is associated with a file to read from.
 void put(Token token)
          Push back a token into the stream.
 void skipSpaces()
          Setter for skipSpaces.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

TokenStreamBaseImpl

public TokenStreamBaseImpl(boolean isFile)
Creates a new object.

Parameters:
isFile - indicator whether or not the token stream is assigned to a file

TokenStreamBaseImpl

public TokenStreamBaseImpl(boolean isFile,
                           Tokens tokens)
Creates a new object.

Parameters:
isFile - indicator whether or not the token stream is assigned to a file
tokens - the tokens to push to the stream initially
Method Detail

closeFileStream

public boolean closeFileStream()
Description copied from interface: TokenStream
Close this stream if it is a file stream.

Specified by:
closeFileStream in interface TokenStream
Returns:
true if the closing was successful
See Also:
TokenStream.closeFileStream()

get

public Token get(TokenFactory factory,
                 Tokenizer tokenizer)
          throws ScannerException
Description copied from interface: TokenStream
Get the next token from the token stream. If tokens are on the push-back stack then those are delivered otherwise new tokens might be extracted utilizing the token factory and the tokenizer.

Specified by:
get in interface TokenStream
Parameters:
factory - the token factory
tokenizer - the tokenizer
Returns:
the next Token or null if no more tokens are available
Throws:
ScannerException - in case of an error
See Also:
TokenStream.get( de.dante.extex.scanner.type.token.TokenFactory, de.dante.extex.interpreter.Tokenizer)

getLocator

public Locator getLocator()
Description copied from interface: TokenStream
Getter for the locator. The locator describes the place the tokens have been read from in terms of the user. This information is meant for the end user to track down problems.

Specified by:
getLocator in interface TokenStream
Returns:
the locator
See Also:
TokenStream.getLocator()

getNext

protected Token getNext(TokenFactory factory,
                        Tokenizer tokenizer)
                 throws ScannerException
Get the next token when the stack is empty. This method is meant to be overloaded by derived classes.

Parameters:
factory - the factory for new tokens
tokenizer - the classifies for characters
Returns:
the next Token or null
Throws:
ScannerException - in case of an error

isEof

public boolean isEof()
              throws ScannerException
Test for end of file.

Specified by:
isEof in interface TokenStream
Returns:
true iff the stream is at its end
Throws:
ScannerException - in case of an error

isEol

public boolean isEol()
              throws ScannerException
Description copied from interface: TokenStream
Check to see if the token stream is currently at the end of line.

Specified by:
isEol in interface TokenStream
Returns:
true if the stream is at end of line
Throws:
ScannerException - in case that an error has been encountered. Especially if an IO exceptions occurs it is delivered as chained exception in a ScannerException.
See Also:
TokenStream.isEol()

isFileStream

public boolean isFileStream()
Description copied from interface: TokenStream
Check whether the current stream is associated with a file to read from.

Specified by:
isFileStream in interface TokenStream
Returns:
true if the stream is a file stream
See Also:
TokenStream.isFileStream()

put

public void put(Token token)
Description copied from interface: TokenStream
Push back a token into the stream. If the token is null then nothing happens: a null token is not pushed!

Note that it is up to the implementation to accept tokens not produced with the token factory for push back. In general the behaviour in such a case is not defined and should be avoided.

Specified by:
put in interface TokenStream
Parameters:
token - the token to push back
See Also:
TokenStream.put( de.dante.extex.scanner.type.token.Token)

skipSpaces

public void skipSpaces()
Setter for skipSpaces.