From html5-parser

html5-parser latest releasehtml5-parser supported Pythonshtml5-parser licensehtml5-parser monthly downloadshtml5-parser total downloadshtml5-parser GitHub starshtml5-parser last commit

html5-parser wraps gumbo, the C WHATWG parser, and hands the result back as an lxml/ElementTree tree. It is a parse-only front end: gumbo tokenizes and tree-builds in C, then the nodes are copied into whatever backend you pick with treebuilder (lxml by default, or lxml_html, etree, dom, soup). Everything after the parse — querying, mutation, serialization — is the chosen backend’s job, not html5-parser’s. It ships in calibre and is a common drop-in wherever a WHATWG-correct tree is wanted inside an existing lxml pipeline.

turbohtml covers the same ground with a single library: it parses the same WHATWG tree in its own C engine, then keeps you inside a fully typed Document for querying, editing, and serialization, with no libxml2 or gumbo build dependency to carry.

turbohtml vs html5-parser

Dimension

turbohtml

html5-parser

Scope

Parse, query, mutate, and serialize in one library

Parse only; the returned tree is handed to lxml/etree/dom/soup for everything else

Feature breadth

CSS select(), XPath 1.0 xpath(), the find()/find_all() grammar, a full edit surface, Markdown/plain-text renderers, sanitizer, linkifier, structured-data extraction

Whatever the selected backend exposes (lxml’s XPath/cssselect, ElementTree’s API, BeautifulSoup’s, minidom’s)

Performance

Native C engine straight into the native tree; see the table below

Native gumbo parse copied into a libxml2/backend tree

Typing

Fully type annotated with bundled stubs

Untyped; static types come from the chosen backend (e.g. lxml-stubs)

Dependencies

Self-contained C extension, no libxml2/gumbo

Links libxml2 and bundles gumbo; the default path also needs lxml

Maintenance

Actively developed

Stable and maintained by the calibre author, low churn

Feature overlap

Both are native WHATWG parsers with no pure-Python pass, so the parse call and the tree walk port directly:

  • html5_parser.parse(markup) for str or bytes input maps to turbohtml.parse().

  • Encoding control (transport_encoding/fallback_encoding) maps to the encoding and detect_encoding arguments of turbohtml.parse().

  • The root element (return_root=True, the default) is doc.root.

  • XPath 1.0 querying is at parity: root.xpath(...) maps to xpath(), and both stop at XPath 1.0 with EXSLT (libxml2 has no XPath 2.0/XQuery either).

  • Element accessors port exactly as in the From lxml section, since html5-parser returns lxml’s tree: el.get/el.attrib become attrs, el.getparent() becomes parent.

What turbohtml adds

  • CSS selection built in: select()/select_one() plus the find()/find_all() filter grammar, with no lxml.cssselect detour.

  • A full mutation surface on the typed tree, so edits do not require pulling in libxml2 semantics.

  • Serialization in the same library: the html property, encode(), and the Markdown/PlainText/Html renderers.

  • Text extraction as first-class API: text, strings, stripped_strings.

  • A real <template> content document, and source positions built in via parse(..., positions=True).

  • Higher-level extraction that html5-parser leaves to your pipeline: the sanitizer, the linkifier, and structured-data parsing.

  • Full static typing across the tree, where html5-parser is untyped.

What html5-parser has that turbohtml does not

  • Pluggable treebuilders. treebuilder='etree', 'dom', or 'soup' returns a stdlib ElementTree, xml.dom.minidom, or BeautifulSoup tree directly. turbohtml returns only its own Document; to feed one of those ecosystems you serialize with html and reparse in the target library.

  • The libxml2 stack on the returned tree. Because the default output is an lxml element, XSLT, DTD/RelaxNG/XML-Schema validation, and C14N are one call away. turbohtml has no equivalent for these; only XPath 1.0 is at parity.

  • Namespaced XHTML output. maybe_xhtml and namespace_elements produce libxml2 namespace-prefixed nodes for XHTML and foreign content. turbohtml builds the WHATWG HTML tree (SVG/MathML foreign content included) but does not expose it as namespaced libxml2 elements.

  • Line numbers as attributes. line_number_attr writes each element’s source line into an attribute on the tree itself. turbohtml records source positions with positions=True, but as node metadata, not as injected attributes.

Performance

html5-parser builds its tree through gumbo into libxml2, where turbohtml runs its own C engine straight into the native tree, so parsing the same document is more than an order of magnitude faster:

parse

turbohtml

html5-parser

4 kB document

2.8 µs

64.1 µs (22.9x)

92 kB document

130 µs

1.84 ms (14.2x)

3 MB document

4.63 ms

59.7 ms (12.9x)

How to migrate

# html5-parser
from html5_parser import parse

root = parse(markup)  # an lxml.etree element

Swap the import for turbohtml’s parse(), which returns a Document instead of a bare root element:

html5-parser

turbohtml

parse(markup)

turbohtml.parse()

parse(markup, return_root=True) (root element)

doc.root

parse(markup, return_root=False) (ElementTree)

the Document itself

parse(markup, transport_encoding=..., fallback_encoding=...)

parse(markup, encoding=..., detect_encoding=...)

root.xpath("//td")

xpath()

root.cssselect("td") (via lxml)

select(), select_one()

el.get(...) / el.attrib

attrs

el.text / el.tail

child Text nodes, or the text property

el.getparent()

parent

treebuilder='etree' | 'dom' | 'soup'

no equivalent; serialize with html and reparse in the target library

 
from turbohtml import parse

doc = parse("<table><tr><td>cell</td></table>")
print(doc.find("td").text)  # the tbody the WHATWG algorithm inserts is walked the same way

cell

Gotchas and pitfalls

  • Return type. html5-parser hands back a bare lxml root element (or an ElementTree with return_root=False); turbohtml hands back a Document. Reach the root through doc.root.

  • The text/tail model. lxml’s two string fields (.text, .tail) become real Text child nodes. Read a subtree’s visible text through the text property instead of stitching the pair together.

  • The rest of the libxml2 stack is gone on purpose. Dropping the libxml2/gumbo build dependency also drops XSLT, DTD/RelaxNG/XML-Schema validation, and C14N. If a pipeline depends on those, keep html5-parser for that stage.

  • Encoding. html5-parser distinguishes transport_encoding (an authoritative label) from fallback_encoding (a guess). turbohtml’s encoding is the explicit label and detect_encoding=True opts into sniffing; there is no separate fallback slot.

  • Namespaces. Without maybe_xhtml/namespace_elements there is nothing to port; with them, expect the turbohtml tree to be plain WHATWG HTML rather than namespace-prefixed libxml2 nodes.