kmy.jint.parser
Class ReaderTokenizer

java.lang.Object
  |
  +--kmy.jint.parser.AbstractTokenizer
        |
        +--kmy.jint.parser.ReaderTokenizer

public class ReaderTokenizer

extends AbstractTokenizer

implements ErrorCodes

Implementation of AbstractTokenizer that reads Jint tokens from a Reader. Jint tokens are basically the same as Java tokens, except for text processing tokens (like regular expression), '~=' token and '::' token (which is not currently used).

Field Summary

(package private) static int	BUF_SIZE
(package private) char[]	buffer
(package private) boolean	eofDetected
(package private) java.io.Reader	in
(package private) int	index
(package private) boolean	nestedCommentsEnabled
(package private) int	rFilePos
(package private) int	size
(package private) boolean	slashWillBeRegex If this flag is set and the next character is '/' it will be treated as an opening character of regex.
(package private) static int	SUPER_BIT
(package private) static int	UNGET_SIZE

Fields inherited from class kmy.jint.parser.AbstractTokenizer

dVal, filePos, ident, lVal, nameTable, sVal, sVal2, tType

Fields inherited from interface kmy.jint.constants.ErrorCodes

ERR_A_ARREMPTY, ERR_A_ARRNOTEMPTY, ERR_A_BADDCL, ERR_A_BADMETHODNAME, ERR_A_BADOP, ERR_A_BADSWITCHTYPE, ERR_A_BOOLCAST, ERR_A_CANTACCLABEL, ERR_A_CANTCAST, ERR_A_CANTMAKEARR, ERR_A_CANTRESOLVE, ERR_A_CONSTERROR, ERR_A_EXPLCONTEXT, ERR_A_FIELDNOTIDENT, ERR_A_FIELDNOTSTAT, ERR_A_IMPLICITDCL, ERR_A_INSTOFPRIM1, ERR_A_INSTOFPRIM2, ERR_A_INTERNAL, ERR_A_LABELREDEF, ERR_A_MATCHARG1, ERR_A_MATCHARG2, ERR_A_MUSTBEBOOL, ERR_A_NEVER, ERR_A_NEWARRARGS, ERR_A_NOBESTMATCH, ERR_A_NOCONSTRUCTOR, ERR_A_NOFIELD, ERR_A_NOMETHOD, ERR_A_NONSTATICACC, ERR_A_NOOUTERCLASS, ERR_A_NOTCAUGHT, ERR_A_NOTINDEXABLE, ERR_A_NOTINTARGS, ERR_A_NOTINTEGER, ERR_A_NOTLVALUE, ERR_A_NOTNUMBER, ERR_A_NOTSIMPLEID, ERR_A_NOTTHROWABLE, ERR_A_NOTTHROWN, ERR_A_NOVAR, ERR_A_OBJTOPRIMCAST, ERR_A_PARAMUNRES, ERR_A_PRIMSYNC, ERR_A_PRIMTOOBJCAST, ERR_A_REGEXTE, ERR_A_STATICTHIS, ERR_A_TOBOOLCAST, ERR_A_TOVOIDCAST, ERR_A_TYPEMISMATCH, ERR_A_TYPENOTFOUND, ERR_A_VARREDECL, ERR_A_VOIDCAST, ERR_C_BADOP, ERR_C_CANTASSIGN, ERR_C_CASEMISMATCH, ERR_C_CVTNULLTOPRIM, ERR_C_DUPCASE, ERR_C_GENERIC, ERR_C_INTERFCONSTR, ERR_C_INTERNAL, ERR_C_MATCHARG1, ERR_C_NESTING, ERR_C_NEVER, ERR_C_NOCNSTRWOPRM, ERR_C_NOENCLOSING, ERR_C_NONSTATINIT, ERR_C_NOTBOOL, ERR_C_PICKEDNONSTR, ERR_C_PRIMELEM, ERR_C_RETURNREQ, ERR_C_THISASSIGN, ERR_C_TWODEFAULT, ERR_L_BACKSLASHEOF, ERR_L_CHAREND, ERR_L_COMMENTEND, ERR_L_EMPTYCHAR, ERR_L_EOFINCHAR, ERR_L_EOFINREGEX, ERR_L_EOFINSTRING, ERR_L_ILLEGALCHAR, ERR_L_IOERROR, ERR_L_NEVER, ERR_L_NUMFORMAT, ERR_L_REGEXEND, ERR_L_STRINGEND, ERR_P_ARRNOTEMPTY, ERR_P_CASEEXPECTED, ERR_P_CATCHEXPECTED, ERR_P_CLASSKEYWORD, ERR_P_DUPEXTENDS, ERR_P_DUPIMPLEMENTS, ERR_P_IDENTEXPECTED, ERR_P_NOTIMPL, ERR_P_NOTSTANDALONE, ERR_P_OPEXPECTED, ERR_P_REGEXEXPECTED, ERR_P_TOKENEXPECTED, ERR_P_TOKENINSERTED, ERR_P_TYPEEXPECTED, ERR_P_UNEXPECTED, ERR_P_VALUEEXPECTED, ERR_P_VAREXPECTED, ERR_T_ARRNOTEMPTY, ERR_T_CANTRESOLVE, ERR_T_IDEXPECTED, ERR_T_INTERNAL, ERR_T_MULTIINHERIT, ERR_T_NOTLIST, ERR_T_NOTSIMPLEID, ERR_T_ONFINISH

Fields inherited from interface kmy.jint.constants.MiniErrorCodes

ERR_F_BADEND, ERR_F_BADFORMATCHAR, ERR_F_GENERIC, ERR_F_INCOMPLETE, ERR_F_NOCURLBRACKET, ERR_F_NOTWITHJAVA, ERR_F_NOVARNAME, ERR_MASK_CLASSTABLE, ERR_MASK_COMPILER, ERR_MASK_ERROR, ERR_MASK_FATAL, ERR_MASK_FORMAT, ERR_MASK_GENERATOR, ERR_MASK_INFO, ERR_MASK_INTERNAL, ERR_MASK_LEXER, ERR_MASK_MODULE, ERR_MASK_PARSER, ERR_MASK_REGEX, ERR_MASK_SEVERITY, ERR_MASK_TYPER, ERR_MASK_VERBOSE, ERR_MASK_WARNING, ERR_R_BADBACKREF, ERR_R_BADSTART, ERR_R_BQBAD, ERR_R_BQUNFINISHED, ERR_R_CTLINVALID, ERR_R_CTLUNFINISHED, ERR_R_EXTRABRACKET, ERR_R_GENERIC, ERR_R_HEXBADNUMBER, ERR_R_HEXUNFINISHED, ERR_R_INTERNAL, ERR_R_NOBRACKET, ERR_R_NOCURLBRACKET, ERR_R_NOSQBRACKET, ERR_R_NOVAREXPR, ERR_R_NOVARNAME, ERR_R_STRAYBSLASH

Constructor Summary

ReaderTokenizer(java.io.Reader in, NameTable nt)
Creates a new ReaderTokenizer given a Reader stream and a NameTable.

Method Summary

void	close() Frees resources held by this stream.
private void	error(int code, java.lang.Object info)
(package private) void	fillBuffer()
static void	main(java.lang.String[] args)
private int	nextChar()
private void	nextLine()
private int	nextRawChar()
private int	nextRegChar() Read char in regex
void	nextToken() Reads new token from the stream.
private void	prevLine()
private void	readCharacter()
private int	readHexChar(int max)
private void	readNumber(char c0)
private int	readOctalChar()
private void	readRegex(char t, char delim) Read regex, substitution, transliteration or file pattern
private int	readRegOpt()
private java.lang.String	readRegSpan(char delim)
private void	readSpecial(char c0)
private void	readString()
private void	readWord(char c0)
(package private) void	skipSlashSlashComment()
(package private) void	skipSlashStarComment()
(package private) void	ungetChar(int c)

Methods inherited from class kmy.jint.parser.AbstractTokenizer

getColumn, getLine, makeFilePos

Methods inherited from class java.lang.Object

, clone, equals, finalize, getClass, hashCode, notify, notifyAll, registerNatives, toString, wait, wait, wait

Field Detail

SUPER_BIT

static final int SUPER_BIT

BUF_SIZE

static final int BUF_SIZE

UNGET_SIZE

static final int UNGET_SIZE

in

java.io.Reader in

buffer

char[] buffer

index

int index

size

int size

rFilePos

int rFilePos

eofDetected

boolean eofDetected

nestedCommentsEnabled

boolean nestedCommentsEnabled

slashWillBeRegex

boolean slashWillBeRegex

If this flag is set and the next character is '/' it will be treated as an opening character of regex.

Constructor Detail

ReaderTokenizer

public ReaderTokenizer(java.io.Reader in,
                       NameTable nt)

Creates a new ReaderTokenizer given a Reader stream and a NameTable. Most of the time one has to create a single NameTable and pass it to all ReaderTokenizers.

Method Detail

error

private final void error(int code,
                         java.lang.Object info)

fillBuffer

void fillBuffer()

nextLine

private final void nextLine()

prevLine

private void prevLine()

ungetChar

void ungetChar(int c)

nextRawChar

private final int nextRawChar()

readHexChar

private final int readHexChar(int max)

readOctalChar

private final int readOctalChar()

nextChar

private final int nextChar()

nextRegChar

private final int nextRegChar()

Read char in regex

readString

private void readString()

readCharacter

private void readCharacter()

skipSlashSlashComment

void skipSlashSlashComment()

skipSlashStarComment

void skipSlashStarComment()

readSpecial

private void readSpecial(char c0)

readNumber

private void readNumber(char c0)

readWord

private void readWord(char c0)

readRegOpt

private int readRegOpt()

readRegSpan

private java.lang.String readRegSpan(char delim)

readRegex

private void readRegex(char t,
                       char delim)

Read regex, substitution, transliteration or file pattern

nextToken

public void nextToken()

Description copied from class: AbstractTokenizer

Reads new token from the stream. Stores token type into tType field and possibly initializes other fields (depending on tType value).

Overrides:

nextToken in class AbstractTokenizer

close

public void close()

Description copied from class: AbstractTokenizer

Frees resources held by this stream. Method nextToken() must not be called after call to close.

Overrides:

close in class AbstractTokenizer

main

public static void main(java.lang.String[] args)