all Python 3codecsterm

codecs.BOM_UTF8

byte order marks (BOM)

The bytes b'\xef\xbb\xbf', the UTF-8 signature at the start of a file. Excel and Windows Notepad write it; the 'utf-8' decoder keeps it, 'utf-8-sig' strips it.

Syntax

codecs.BOM_UTF8  # b'\xef\xbb\xbf'

The classic CSV trap: the first header reads as '\ufeffid', and row['id'] raises KeyError although the file looks fine in an editor. The cure is open(..., encoding='utf-8-sig'): it strips a BOM if present and reads a BOM-less file as usual. When writing a CSV for Excel the same 'utf-8-sig' adds the BOM so Cyrillic opens without mojibake.

Examples

import codecs
data = codecs.BOM_UTF8 + 'id;name'.encode('utf-8')
print(repr(data.decode('utf-8').split(';')[0]))  # → '\ufeffid'
print(data.decode('utf-8-sig').split(';')[0])  # → id
print(data.startswith(codecs.BOM_UTF8))  # → True

See also

Open in the full glossarydocs.python.orgReport an inaccuracy