From html5-parser

html5-parser latest releasehtml5-parser supported Pythonshtml5-parser licensehtml5-parser monthly downloadshtml5-parser total downloadshtml5-parser GitHub starshtml5-parser last commit

html5-parser wraps gumbo, the C WHATWG parser, and hands the result back as an lxml/ElementTree tree. It is a parse-only front end: gumbo tokenizes and tree-builds in C, then the nodes are copied into whatever backend you pick with treebuilder (lxml by default, or lxml_html, etree, dom, soup). Everything after the parse — querying, mutation, serialization — is the chosen backend’s job, not html5-parser’s. It ships in calibre and is a common drop-in wherever a WHATWG-correct tree is wanted inside an existing lxml pipeline.

turbohtml covers the same ground with a single library: it parses the same WHATWG tree in its own C engine, then keeps you inside a fully typed Document for querying, editing, and serialization, with no libxml2 or gumbo build dependency to carry.

turbohtml vs html5-parser

Dimension

turbohtml

html5-parser

Scope

Parse, query, mutate, and serialize in one library

Parse only; the returned tree is handed to lxml/etree/dom/soup for everything else

Feature breadth

CSS select(), XPath 1.0 xpath(), the find()/find_all() grammar, a full edit surface, Markdown/plain-text renderers, sanitizer, linkifier, structured-data extraction

Whatever the selected backend exposes (lxml’s XPath/cssselect, ElementTree’s API, BeautifulSoup’s, minidom’s)

Performance

Native C engine straight into the native tree; see the table below

Native gumbo parse copied into a libxml2/backend tree

Typing

Fully type annotated with bundled stubs

Untyped; static types come from the chosen backend (e.g. lxml-stubs)

Dependencies

Self-contained C extension, no libxml2/gumbo

Links libxml2 and bundles gumbo; the default path also needs lxml

Maintenance

Actively developed

Stable and maintained by the calibre author, low churn

Feature overlap

Both are native WHATWG parsers with no pure-Python pass, so the parse call and the tree walk port directly:

  • html5_parser.parse(markup) for str or bytes input maps to turbohtml.parse().

  • Encoding control (transport_encoding/fallback_encoding) maps to the encoding and detect_encoding arguments of turbohtml.parse().

  • The root element (return_root=True, the default) is doc.root.

  • XPath 1.0 querying is at parity: root.xpath(...) maps to xpath(), and both stop at XPath 1.0 with EXSLT (libxml2 has no XPath 2.0/XQuery either).

  • Element accessors port exactly as in the From lxml section, since html5-parser returns lxml’s tree: el.get/el.attrib become attrs, el.getparent() becomes parent.

What turbohtml adds

  • CSS selection built in: select()/select_one() plus the find()/find_all() filter grammar, with no lxml.cssselect detour.

  • A full mutation surface on the typed tree, so edits do not require pulling in libxml2 semantics.

  • Serialization in the same library: the html property, encode(), and the Markdown/PlainText/Html renderers.

  • Text extraction as first-class API: text, strings, stripped_strings.

  • A real <template> content document, and source positions built in via parse(..., positions=True).

  • Higher-level extraction that html5-parser leaves to your pipeline: the sanitizer, the linkifier, and structured-data parsing.

  • Full static typing across the tree, where html5-parser is untyped.

What html5-parser has that turbohtml does not

  • Pluggable treebuilders. treebuilder='etree', 'dom', or 'soup' returns a stdlib ElementTree, xml.dom.minidom, or BeautifulSoup tree directly. turbohtml returns only its own Document; to feed one of those ecosystems you serialize with html and reparse in the target library.

  • The libxml2 stack on the returned tree. Because the default output is an lxml element, XSLT, DTD/RelaxNG/XML-Schema validation, and C14N are one call away. turbohtml has no equivalent for these; only XPath 1.0 is at parity.

  • Namespaced XHTML output. maybe_xhtml and namespace_elements produce libxml2 namespace-prefixed nodes for XHTML and foreign content. turbohtml builds the WHATWG HTML tree (SVG/MathML foreign content included) but does not expose it as namespaced libxml2 elements.

  • Line numbers as attributes. line_number_attr writes each element’s source line into an attribute on the tree itself. turbohtml records source positions with positions=True, but as node metadata, not as injected attributes.

Performance

html5-parser builds its tree through gumbo into libxml2, where turbohtml runs its own C engine straight into the native tree, so parsing the same document runs 6x to 12x faster:

parse

turbohtml

html5-parser

wpt tiny (0.6 kB)

1.21 µs

12 µs (10.0x)

wpt small (4 kB)

9.58 µs

68.5 µs (7.2x)

wpt medium (9.6 kB)

24.1 µs

163 µs (6.8x)

wpt large (92 kB)

209 µs

2.13 ms (10.2x)

wpt CJK (124 kB)

408 µs

2.53 ms (6.2x)

whatwg spec (235 kB)

400 µs

4.87 ms (12.2x)

How to migrate

# html5-parser
from html5_parser import parse

root = parse(markup)  # an lxml.etree element

Swap the import for turbohtml’s parse(), which returns a Document instead of a bare root element:

html5-parser

turbohtml

parse(markup)

turbohtml.parse()

parse(markup, return_root=True) (root element)

doc.root

parse(markup, return_root=False) (ElementTree)

the Document itself

parse(markup, transport_encoding=..., fallback_encoding=...)

parse(markup, encoding=..., detect_encoding=...)

root.xpath("//td")

xpath()

root.cssselect("td") (via lxml)

select(), select_one()

el.get(...) / el.attrib

attrs

el.text / el.tail

child Text nodes, or the text property

el.getparent()

parent

treebuilder='etree' | 'dom' | 'soup'

no equivalent; serialize with html and reparse in the target library

 
from turbohtml import parse

doc = parse("<table><tr><td>cell</td></table>")
print(doc.find("td").text)  # the tbody the WHATWG algorithm inserts is walked the same way

cell

Gotchas and pitfalls

  • Return type. html5-parser hands back a bare lxml root element (or an ElementTree with return_root=False); turbohtml hands back a Document. Reach the root through doc.root.

  • The text/tail model. lxml’s two string fields (.text, .tail) become real Text child nodes. Read a subtree’s visible text through the text property instead of stitching the pair together.

  • The rest of the libxml2 stack is gone on purpose. Dropping the libxml2/gumbo build dependency also drops XSLT, DTD/RelaxNG/XML-Schema validation, and C14N. If a pipeline depends on those, keep html5-parser for that stage.

  • Encoding. html5-parser distinguishes transport_encoding (an authoritative label) from fallback_encoding (a guess). turbohtml’s encoding is the explicit label and detect_encoding=True opts into sniffing; there is no separate fallback slot.

  • Namespaces. Without maybe_xhtml/namespace_elements there is nothing to port; with them, expect the turbohtml tree to be plain WHATWG HTML rather than namespace-prefixed libxml2 nodes.