From the standard library¶

Python’s standard library ships HTML primitives in the html package. html.escape() and html.unescape() handle entity encoding and decoding, html.entities exposes the reference tables, and html.parser.HTMLParser is a SAX-style tokenizer you subclass and drive with handle_* callbacks. These are the zero-dependency, always-available building blocks that ship with CPython; countless scripts, templating helpers, and scrapers reach for them because they are already installed. The scope stops at tokenizing and entity work: html.parser does not build a document tree, does not implement WHATWG error recovery, and is explicitly documented as not fully HTML5-conformant.

turbohtml covers that same ground and extends past it. turbohtml.escape() and turbohtml.unescape() match the stdlib functions byte for byte, turbohtml.tokenize() and turbohtml.Tokenizer replace the callback tokenizer, and turbohtml.migration.stdlib.HTMLParser keeps your existing handle_* subclass working unchanged. Everything runs over a WHATWG-conformant C core that also builds a full parse tree, which html.parser has no equivalent for.

turbohtml vs stdlib¶

Dimension	turbohtml	stdlib (`html`)
Scope	Escape/unescape, tokenizer, and full WHATWG tree construction	Escape/unescape, entity tables, and a tokenizer only (no tree)
Feature breadth	Tokens, tree, selectors, serialization, plus the callback shim	`escape`/`unescape`, `html.entities` tables, `HTMLParser` callbacks
Performance	SIMD scanning, several times faster on escape/unescape	Pure-Python entity scan and tokenizer
Typing	Fully type annotated across the public surface	Annotated in typeshed stubs, not conformant behavior
Dependencies	Compiled C extension (wheels), installed from PyPI	Built into CPython, zero install
Maintenance	Actively developed, tracks the WHATWG spec	Stable CPython module, `html.parser` frozen as non-conformant

Feature overlap¶

The shared surface ports one-to-one:

html.escape() → turbohtml.escape(), same signature and output.
html.unescape() → turbohtml.unescape(), same signature and output.
html.parser.HTMLParser subclasses → turbohtml.migration.stdlib.HTMLParser, same handle_* callbacks and feed/close/reset/getpos methods.

What turbohtml adds¶

WHATWG-conformant tokenizing and tree construction via turbohtml.parse() and turbohtml.parse_fragment(); html.parser tokenizes but never builds a tree and is documented as not HTML5-conformant.
A token stream you drive yourself through turbohtml.tokenize() and turbohtml.Tokenizer, instead of inverting control into callbacks.
Verbatim source capture per token (capture_source=True → token.source) and unresolved reference tokens (resolve_references=False → TokenType.CHARACTER_REFERENCE).
SIMD-accelerated escape/unescape scanning.

What stdlib has that turbohtml does not¶

html.parser and the html functions are built into CPython with no install step. turbohtml ships a compiled extension from PyPI; in environments that cannot install wheels or build C, the stdlib remains the only option.
html.entities exposes the raw reference tables (name2codepoint, codepoint2name, html5) as public data. turbohtml resolves references through escape/unescape and the tokenizer rather than exposing the dicts; if you consume those tables directly, keep importing html.entities.

Performance¶

operation	turbohtml	stdlib
escape — tiny plain (64 B)	56.2 ns	117 ns (2.1x)
escape — medium markup (4 KiB)	2.21 µs	7.31 µs (3.4x)
escape — no-op prose (4 MiB)	116 µs	2.54 ms (22.0x)
escape — book text (3 MiB)	674 µs	2.65 ms (4.0x)
escape — book HTML (4 MiB)	1.22 ms	4.74 ms (3.9x)
escape — spec HTML, dense (4 MiB)	4.92 ms	13 ms (2.7x)
escape — UCS-2 plain (4 MiB)	803 µs	2.62 ms (3.3x)
escape — UCS-2 markup (4 MiB)	5.7 ms	11.2 ms (2.0x)
escape — UCS-4 plain (4 MiB)	920 µs	5.73 ms (6.3x)
escape — UCS-4 markup (4 MiB)	6.69 ms	20 ms (3.0x)
unescape — tiny plain (64 B)	34 ns	37.7 ns (1.2x)
unescape — medium dense refs (4 KiB)	7.09 µs	70.7 µs (10.0x)
unescape — numeric refs (4 KiB)	4.97 µs	79.2 µs (16.0x)
unescape — book HTML, real refs (4 MiB)	3.34 ms	8.72 ms (2.7x)
unescape — escaped book HTML (5 MiB)	1.76 ms	25.3 ms (14.4x)
unescape — dense refs (4 MiB)	9.67 ms	85.7 ms (8.9x)
unescape — UCS-2 refs (4 MiB)	2.75 ms	19.1 ms (7.0x)
tokenize — typical markup	28.5 µs	441 µs (15.5x)
tokenize — text-heavy prose	535 ns	2.74 µs (5.2x)
tokenize — attribute-heavy	18.2 µs	286 µs (15.8x)
tokenize — script-heavy	11.5 µs	152 µs (13.2x)
tokenize — entity-heavy	19.8 µs	192 µs (9.8x)
tokenize — wpt tiny (0.6 kB)	1.52 µs	18.4 µs (12.1x)
tokenize — wpt small (4 kB)	11.9 µs	167 µs (14.0x)
tokenize — wpt medium (9.6 kB)	27.3 µs	387 µs (14.2x)
tokenize — wpt large (92 kB)	314 µs	3.9 ms (12.5x)
tokenize — wpt CJK (124 kB)	550 µs	8.58 ms (15.7x)
tokenize — whatwg spec (235 kB)	624 µs	7.42 ms (12.0x)
tokenize — ecmascript spec (3 MB)	6.2 ms	53.9 ms (8.7x)
tokenize — whatwg spec source (7.9 MB)	36.5 ms	370 ms (10.2x)
feed and dispatch a page — daring fireball (10 kB)	85.9 µs	312 µs (3.7x)
feed and dispatch a page — ars technica (56 kB)	372 µs	1.41 ms (3.8x)
feed and dispatch a page — mozilla blog (95 kB)	809 µs	3.2 ms (4.0x)
feed and dispatch a page — whatwg spec (235 kB)	2.44 ms	7.19 ms (3.0x)

turbohtml.escape() and turbohtml.unescape() reproduce the standard-library functions byte for byte, so they are drop-ins, but scan with SIMD and run several times faster.

How to migrate¶

Swap the imports and, if you subclass the parser, swap the base class:

stdlib call	turbohtml call
`html.escape(s)`	`turbohtml.escape(s)`
`html.unescape(s)`	`turbohtml.unescape(s)`
`class P(html.parser.HTMLParser)`	`class P(turbohtml.migration.stdlib.HTMLParser)`
`handle_starttag(tag, attrs)`	`token.type is TokenType.START_TAG` → `token.tag`, `token.attrs`
`handle_startendtag(tag, attrs)`	`TokenType.START_TAG` with `token.self_closing`
`handle_endtag(tag)`	`TokenType.END_TAG` → `token.tag`
`handle_data(data)`	`TokenType.TEXT` → `token.data`
`handle_comment(data)`	`TokenType.COMMENT` → `token.data`
`handle_decl(decl)`	`TokenType.DOCTYPE` → `token.name`
`handle_entityref`/`handle_charref`	`tokenize(..., resolve_references=False)` → `TokenType.CHARACTER_REFERENCE`, else resolved in `token.data`
`get_starttag_text()`	`tokenize(..., capture_source=True)` → `token.source`

Escape and unescape are literal drop-ins:

import html
from turbohtml import escape, unescape

print(escape('<a href="x">') == html.escape('<a href="x">'))
print(unescape("caf&eacute; &#127881;") == html.unescape("caf&eacute; &#127881;"))

True
True

To keep an existing html.parser.HTMLParser subclass, swap its base class for turbohtml.migration.stdlib.HTMLParser: the same handle_* callbacks and feed/close methods run over the WHATWG-conformant tokenizer. Or drop the subclass and take the token stream from turbohtml.tokenize() (or turbohtml.Tokenizer.feed() for incremental input), or skip tokens entirely and turbohtml.parse() straight to a tree. All three are WHATWG-conformant, unlike html.parser. The How-to guides guide has a worked port.

HTMLParser is a SAX-style callback API; turbohtml gives you the events as a token stream you drive yourself, which inverts the control flow. Each handle_* override becomes a branch on Token.type:

import turbohtml
from turbohtml import TokenType

events = []
for token in turbohtml.tokenize('<p class="x">Hi &amp; bye</p>'):
    if token.type is TokenType.START_TAG:
        events.append(("start", token.tag, token.attrs))
    elif token.type is TokenType.TEXT:
        events.append(("data", token.data))
    elif token.type is TokenType.END_TAG:
        events.append(("end", token.tag))
print(events)

[('start', 'p', [('class', 'x')]), ('data', 'Hi & bye'), ('end', 'p')]

Gotchas and pitfalls¶

The token stream inverts html.parser’s callback control flow: you loop over tokens and branch on Token.type instead of overriding handle_* (unless you subclass turbohtml.migration.stdlib.HTMLParser, which keeps the callbacks).
By default token.data already holds decoded text (the equivalent of convert_charrefs=True). To recover the split stream convert_charrefs=False gives, pass resolve_references=False and handle TokenType.CHARACTER_REFERENCE tokens, whose token.source is the verbatim reference and token.data its resolved value. On turbohtml.migration.stdlib.HTMLParser the convert_charrefs argument is accepted for signature compatibility but ignored; references are always resolved.
The verbatim start-tag text get_starttag_text() returns is token.source once you pass capture_source=True; it is not captured by default.
html.parser is documented as not fully HTML5-conformant, so tricky recovery cases (malformed tags, misnested elements, foreign content) can tokenize differently. turbohtml follows the WHATWG spec, so output may diverge from a legacy html.parser run on the same broken input; the turbohtml result is the conformant one.
If your code imports the reference tables from html.entities directly, keep that import: turbohtml does not re-export name2codepoint/codepoint2name/html5.