The token
module in Python comprises tokens that represent the numeric values of leaf nodes of the parse tree.
The different methods in the module are as follows:
tok_name
constantThe tok_name
constant returns a dictionary that maps the numeric values of the constants specified in this module back to name strings, enabling the generation of more human-readable representations of parse trees.
token.tok_name
ISTERMINAL
methodThe ISTERMINAL
method returns True
if the given token is a terminal token. Otherwise, it returns False
.
token.ISTERMINAL(x)
Note: The
x
refers to the given token.
ISNONTERMINAL
methodThe ISNONTERMINAL
method returns True
if the given token is not a terminal token. Otherwise, it returns False
.
token.ISNONTERMINAL(x)
Note: The
x
refers to the given token.
ISEOF
methodThe ISEOF
method returns True
if the given token is a marker token indicating the end of input.
token.ISEOF(x)
Note:
x
refers to the given token.
import tokentk = token.PLUSprint("token.ISTERMINAL(%s) = %s" % (tk, token.ISTERMINAL(tk)))tk = token.COLONprint("token.ISNONTERMINAL(%s) = %s" % (tk, token.ISNONTERMINAL(tk)))tk = token.NEWLINEprint("token.ISEOF(%s) = %s" % (tk, token.ISEOF(tk)))print("token.tok_name = %s" % (token.tok_name,))
token
module.