aplp.TokenMarker Class Reference

Inheritance diagram for aplp.TokenMarker:

List of all members.

Public Member Functions
Token	markTokens (Segment line, int lineIndex)
boolean	supportsMultilineTokens ()
void	insertLines (int index, int lines)
void	deleteLines (int index, int lines)
int	getLineCount ()
boolean	isNextLineRequested ()
Protected Member Functions
abstract byte	markTokensImpl (byte token, Segment line, int lineIndex)
	TokenMarker ()
void	ensureCapacity (int index)
void	addToken (int length, byte id)
Protected Attributes
Token	firstToken
Token	lastToken
LineInfo[]	lineInfo
int	length
int	lastLine
boolean	nextLineRequested
Classes
class	LineInfo

---

Detailed Description

A token marker that splits lines of text into tokens. Each token carries a length field and an indentification tag that can be mapped to a color for painting that token.

For performance reasons, the linked list of tokens is reused after each line is tokenized. Therefore, the return value of markTokens should only be used for immediate painting. Notably, it cannot be cached.

Author:

Slava Pestov

Version:

Id

TokenMarker.java,v 1.1 2004/09/14 10:58:06 cvs-3apl Exp

See also:

org.gjt.sp.jedit.syntax.Token

---

Constructor & Destructor Documentation

aplp.TokenMarker.TokenMarker (   )  [protected]

Creates a new TokenMarker. This DOES NOT create a lineInfo array; an initial call to insertLines() does that.

---

Member Function Documentation

void aplp.TokenMarker.addToken (  int  length, byte  id )  [protected]

Adds a token to the token list. Parameters: length  The length of the token id  The id of the token

void aplp.TokenMarker.deleteLines (  int  index, int  lines )

Informs the token marker that line have been deleted from the document. This removes the lines in question from the lineInfo array. Parameters: index  The first line number lines  The number of lines

void aplp.TokenMarker.ensureCapacity (  int  index  )  [protected]

Ensures that the lineInfo array can contain the specified index. This enlarges it if necessary. No action is taken if the array is large enough already. It should be unnecessary to call this under normal circumstances; insertLine() should take care of enlarging the line info array automatically. Parameters: index  The array index

int aplp.TokenMarker.getLineCount (   )

Returns the number of lines in this token marker.

void aplp.TokenMarker.insertLines (  int  index, int  lines )

Informs the token marker that lines have been inserted into the document. This inserts a gap in the lineInfo array. Parameters: index  The first line number lines  The number of lines

boolean aplp.TokenMarker.isNextLineRequested (   )

Returns true if the next line should be repainted. This will return true after a line has been tokenized that starts a multiline token that continues onto the next line.

Token aplp.TokenMarker.markTokens (  Segment  line, int  lineIndex )

A wrapper for the lower-level markTokensImpl method that is called to split a line up into tokens. Parameters: line  The line lineIndex  The line number

abstract byte aplp.TokenMarker.markTokensImpl (  byte  token, Segment  line, int  lineIndex )  [protected, pure virtual]

An abstract method that splits a line up into tokens. It should parse the line, and call addToken() to add syntax tokens to the token list. Then, it should return the initial token type for the next line. For example if the current line contains the start of a multiline comment that doesn't end on that line, this method should return the comment token type so that it continues on the next line. Parameters: token  The initial token type for this line line  The line to be tokenized lineIndex  The index of the line in the document, starting at 0 Returns: The initial token type for the next line Implemented in aplp.APLTokenMarker.

boolean aplp.TokenMarker.supportsMultilineTokens (   )

Returns if the token marker supports tokens that span multiple lines. If this is true, the object using this token marker is required to pass all lines in the document to the markTokens() method (in turn). The default implementation returns true; it should be overridden to return false on simpler token markers for increased speed.

---

Member Data Documentation

Token aplp.TokenMarker.firstToken [protected]

The first token in the list. This should be used as the return value from markTokens().

int aplp.TokenMarker.lastLine [protected]

The last tokenized line.

Token aplp.TokenMarker.lastToken [protected]

The last token in the list. New tokens are added here. This should be set to null before a new line is to be tokenized.

int aplp.TokenMarker.length [protected]

The number of lines in the model being tokenized. This can be less than the length of the lineInfo array.

LineInfo [] aplp.TokenMarker.lineInfo [protected]

An array for storing information about lines. It is enlarged and shrunk automatically by the insertLines() and deleteLines() methods.

boolean aplp.TokenMarker.nextLineRequested [protected]

True if the next line should be painted.

---

The documentation for this class was generated from the following file:

• D:/3APL_src/tripleAPL/multi/src/aplp/TokenMarker.java

---