Public Member Functions | List of all members
Rose::BinaryAnalysis::SymbolicExpressionParser::TokenStream Class Reference


A stream of tokens.

This container is conceptually infinite size padded with EOF tokens, although in practice it usually holds only one or two tokens at a time. Tokens are consumed by shifting them off the beginning of the stream.

Definition at line 125 of file SymbolicExpressionParser.h.

#include <Rose/BinaryAnalysis/SymbolicExpressionParser.h>

Public Member Functions

 TokenStream (std::istream &input, const std::string &name="input", unsigned lineNumber=1, unsigned columnNumber=0)
 Scan tokens from a character stream. More...
const std::string & name () const
 Name of this input stream. More...
unsigned lineNumber () const
 Current line number. More...
unsigned columnNumber () const
 Current column number. More...
const Tokenoperator[] (size_t idx)
 Returns the specified token without consuming it. More...
void shift (size_t n=1)
 Consume the specified number of tokens. More...
int nextCharacter ()
 Returns the next character. More...
int consumeCharacter ()
 Consume the next character. More...
void consumeWhiteSpace ()
 Skip over characters until a non-white-space character is encountered. More...
int consumeEscapeSequence ()
 Skip over an escape sequence and return the escaped character. More...
void consumeInlineComment ()
 Skip over angle-bracket comments. More...
void consumeWhiteSpaceAndComments ()
 Skip over white space and/or inline comments. More...
std::string consumeTerm ()
 Parse and consume a term. More...
SymbolicExpression::Type consumeType ()
 Parse and consume a type specification. More...
Token scan ()
 Parse and consume the next token. More...

Constructor & Destructor Documentation

Rose::BinaryAnalysis::SymbolicExpressionParser::TokenStream::TokenStream ( std::istream &  input,
const std::string &  name = "input",
unsigned  lineNumber = 1,
unsigned  columnNumber = 0 

Scan tokens from a character stream.

The name parameter is the name to use for this stream in error messages (usually a file name), and the lineNumber and columnNumber are the position of the first character in the stream.

Definition at line 138 of file SymbolicExpressionParser.h.

Member Function Documentation

const std::string& Rose::BinaryAnalysis::SymbolicExpressionParser::TokenStream::name ( ) const

Name of this input stream.

Definition at line 145 of file SymbolicExpressionParser.h.

unsigned Rose::BinaryAnalysis::SymbolicExpressionParser::TokenStream::lineNumber ( ) const

Current line number.

Definition at line 148 of file SymbolicExpressionParser.h.

unsigned Rose::BinaryAnalysis::SymbolicExpressionParser::TokenStream::columnNumber ( ) const

Current column number.

Definition at line 151 of file SymbolicExpressionParser.h.

const Token& Rose::BinaryAnalysis::SymbolicExpressionParser::TokenStream::operator[] ( size_t  idx)

Returns the specified token without consuming it.

void Rose::BinaryAnalysis::SymbolicExpressionParser::TokenStream::shift ( size_t  n = 1)

Consume the specified number of tokens.

int Rose::BinaryAnalysis::SymbolicExpressionParser::TokenStream::nextCharacter ( )

Returns the next character.

Returns the next character of input or EOF without consuming it.

int Rose::BinaryAnalysis::SymbolicExpressionParser::TokenStream::consumeCharacter ( )

Consume the next character.

Advances over the next character, adjusting line and column information. Returns the character consumed, or EOF.

void Rose::BinaryAnalysis::SymbolicExpressionParser::TokenStream::consumeWhiteSpace ( )

Skip over characters until a non-white-space character is encountered.

int Rose::BinaryAnalysis::SymbolicExpressionParser::TokenStream::consumeEscapeSequence ( )

Skip over an escape sequence and return the escaped character.

void Rose::BinaryAnalysis::SymbolicExpressionParser::TokenStream::consumeInlineComment ( )

Skip over angle-bracket comments.

Consumes angle bracket comments like "<this is a comment>". Any right angle brackets that are part of the comment must be escaped. Unescaped angle brackets can be nested like parentheses.

void Rose::BinaryAnalysis::SymbolicExpressionParser::TokenStream::consumeWhiteSpaceAndComments ( )

Skip over white space and/or inline comments.

std::string Rose::BinaryAnalysis::SymbolicExpressionParser::TokenStream::consumeTerm ( )

Parse and consume a term.

A term is a numeric constant or a symbol. Symbols that start with a letter or underscore contain only letters, underscores, and decimal digits. Symbols that start with other than a letter or underscore end at the first unescaped white-space, comment, parenthesis, angle bracket, or square bracket. Escaping the special character with a backslash will cause it to become part of the symbol.

SymbolicExpression::Type Rose::BinaryAnalysis::SymbolicExpressionParser::TokenStream::consumeType ( )

Parse and consume a type specification.

A type specification is a type name in square brackets. If the square brackets contain only an integer then the type is an integer type with the specified width.

Token Rose::BinaryAnalysis::SymbolicExpressionParser::TokenStream::scan ( )

Parse and consume the next token.

Parses and consumes the next token and return it. Returns the special NONE token at end-of-input.

The documentation for this class was generated from the following file: