all Python 3Exceptionsexception

tokenize.TokenError

module exceptions

An error while tokenizing Python source (e.g. an unclosed bracket or string).

Syntax

raise tokenize.TokenError

It means the source ran out mid-construct: an unclosed bracket or triple-quoted string, with the tokenizer hitting end of input inside it. It is not a syntax check — tokenize never parses grammar, so nonsense that still splits into valid tokens passes without complaint. In interactive tools this error usually means "ask for another line", not "reject the input".

Examples

import tokenize
try:
    raise tokenize.TokenError
except tokenize.TokenError as e:
    print(type(e).__name__)   # → TokenError

See also

Open in the full glossarydocs.python.orgReport an inaccuracy