From lxml¶

lxml is the libxml2/libxslt binding that most Python HTML and XML processing has been built on. lxml.html parses documents into ElementTree-style elements with .text/.tail strings, and the wider stack adds XPath, XSLT, RelaxNG/DTD/XML-Schema validation, and C14N. It is the default reach for scraping, feed parsing, and XML pipelines because it wraps a fast, battle-tested C library and exposes the full ElementTree API.

turbohtml covers the HTML side of that ground with a native C core of its own. turbohtml.parse() builds the WHATWG document tree libxml2’s HTML parser does not, returns a fully type annotated Document, and folds XPath 1.0 (with EXSLT), CSS selection, and the find/find_all grammar into one node API instead of separate findall/xpath/cssselect entry points. It does not attempt XSLT, schema validation, or generic XML; it targets browser-accurate HTML parsing and the query/edit/serialize surface around it.

turbohtml vs lxml¶

Dimension	turbohtml	lxml
Scope	WHATWG HTML5 parse, serialize, edit, CSS, XPath 1.0 + EXSLT, link helpers	Generic XML and HTML via libxml2, plus XSLT, schema validation, C14N
Feature breadth	Browser-accurate HTML tree, one node API for XPath/CSS/find, streaming parse, builder	Full ElementTree API, XPath 1.0, XSLT 1.0, DTD/RelaxNG/XML-Schema, iterparse
Performance	Parses two to four times faster than lxml; stays ahead across the operational surface	Mature libxml2 C core; streaming evaluation narrows on multi-megabyte inputs
Typing	Fully type annotated, ships stubs	Partial; relies on third-party stub packages
Dependencies	Self-contained native C extension	Bundles or links libxml2 and libxslt
Maintenance	Newer, WHATWG-spec-driven	Long-established, widely deployed, actively maintained

Feature overlap¶

These port one-to-one from lxml.html/lxml.etree to turbohtml:

Parsing a document (lxml.html.document_fromstring) and a fragment (lxml.html.fromstring).
Element identity and attributes: el.tag, el.get/el.set/el.attrib, the el.classes set operations.
Tree navigation: getparent, getnext, getprevious, iterdescendants, iterancestors, list(el).
Queries: findall and xpath (XPath 1.0), cssselect (CSS), precompiled etree.XPath objects, node-set $variable bindings, namespaces= prefix maps, and custom XPath callables.
The EXSLT re:, set:, str:, math:, and date: function namespaces.
Locator generation (getroottree().getpath), link iteration and rewriting (iterlinks, make_links_absolute, rewrite_links), source positions (sourceline), tree edits (drop_tag, drop_tree), the lxml.builder.E builder, and serialization (lxml.html.tostring).

What turbohtml adds¶

A WHATWG-conformant parse: malformed input lands in the same tree a browser builds, where libxml2’s HTML parser does not follow the HTML5 tree-construction algorithm.
One node API. XPath, CSS, and the find/find_all grammar are methods on every node rather than three separate extension entry points, and a callable or extensions= mapping that returns an Element is marshaled straight back into the evaluator’s node-set.
Built-in EXSLT. The re:, set:, str:, math:, and date: namespaces dispatch in the compiled-C XPath engine with no per-call registration; lxml has to register libexslt and re-resolve the namespace map on each evaluation.
css_path(), a unique CSS-selector locator, which lxml has no equivalent for.
Full type annotations and shipped stubs across the whole surface.

What lxml has that turbohtml does not¶

The wider libxml2 toolchain is a deliberate clean-break scope cut:

XSLT (lxml.etree.XSLT): no equivalent.
Schema validation (DTD, RelaxNG, XML-Schema, Schematron): no equivalent.
C14N canonicalization: no equivalent.
Generic XML parsing and namespaced XML documents: turbohtml targets HTML; use lxml for arbitrary XML.
XPath is at parity, not a gap. Both are XPath 1.0, and both run EXSLT. The only pieces out of scope are the node-synthesizing str:tokenize/str:split and the implicit current-date date: forms.

Performance¶

turbohtml parses two to four times faster than lxml while matching a browser on malformed input, and stays ahead across the operational surface: fragment parsing, CSS selection, text and tree walks, the link helpers, XPath, and the node-path generators.

operation	turbohtml	lxml
build a list (constructors) — 100 rows	55.7 µs	165 µs (3.0x)
build a list (constructors) — 1k rows	566 µs	1.38 ms (2.5x)
build a list (constructors) — 10k rows	6.41 ms	13.7 ms (2.2x)
build a list (terse builders) — 100 rows	117 µs	200 µs (1.8x)
build a list (terse builders) — 1k rows	1.41 ms	2.06 ms (1.5x)
build a list (terse builders) — 10k rows	14.8 ms	21 ms (1.5x)
construct N elements (no serialize) — 100 rows	43.4 µs	95.9 µs (2.3x)
construct N elements (no serialize) — 1k rows	463 µs	919 µs (2.0x)
construct N elements (no serialize) — 10k rows	4.66 ms	9.7 ms (2.1x)
emit a built tree — 100 rows	4.19 µs	33.3 µs (8.0x)
emit a built tree — 1k rows	42.4 µs	325 µs (7.7x)
emit a built tree — 10k rows	420 µs	3.23 ms (7.8x)
parse to a tree — wpt tiny (0.6 kB)	1.21 µs	3.44 µs (2.9x)
parse to a tree — wpt small (4 kB)	9.57 µs	27 µs (2.9x)
parse to a tree — wpt medium (9.6 kB)	24.2 µs	74 µs (3.1x)
parse to a tree — wpt large (92 kB)	207 µs	645 µs (3.2x)
parse to a tree — wpt CJK (124 kB)	407 µs	1.44 ms (3.6x)
parse to a tree — whatwg spec (235 kB)	405 µs	1.27 ms (3.2x)
parse a fragment — table-row fragment (2 kB)	11.4 µs	33.7 µs (3.0x)
find every anchor — daring fireball (10 kB)	375 ns	5.06 µs (13.5x)
find every anchor — ars technica (56 kB)	828 ns	13.4 µs (16.3x)
find every anchor — mozilla blog (95 kB)	1.18 µs	20.4 µs (17.4x)
find every anchor — whatwg spec (235 kB)	1.32 µs	44.8 µs (34.1x)
select div a[href] — daring fireball (10 kB)	639 ns	29.4 µs (46.0x)
select div a[href] — ars technica (56 kB)	1.46 µs	124 µs (85.2x)
select div a[href] — mozilla blog (95 kB)	2.07 µs	803 µs (388x)
select div a[href] — whatwg spec (235 kB)	1.74 µs	1.34 ms (768x)
select div:has(a) — daring fireball (10 kB)	256 ns	14.3 µs (55.9x)
select div:has(a) — ars technica (56 kB)	1.28 µs	27.7 µs (21.7x)
select div:has(a) — mozilla blog (95 kB)	8.97 µs	60.5 µs (6.8x)
select div:has(a) — whatwg spec (235 kB)	6.08 µs	71.2 µs (11.8x)
collect visible text — daring fireball (10 kB)	2.69 µs	3.39 µs (1.3x)
collect visible text — ars technica (56 kB)	13.4 µs	16 µs (1.2x)
collect visible text — mozilla blog (95 kB)	23.3 µs	25.8 µs (1.2x)
collect visible text — whatwg spec (235 kB)	76.8 µs	89.5 µs (1.2x)
serialize a parsed tree — daring fireball (10 kB)	7.14 µs	39.9 µs (5.6x)
serialize a parsed tree — ars technica (56 kB)	40.7 µs	198 µs (4.9x)
serialize a parsed tree — mozilla blog (95 kB)	79.2 µs	432 µs (5.5x)
serialize a parsed tree — whatwg spec (235 kB)	209 µs	835 µs (4.1x)
tag every link rel=nofollow — daring fireball (10 kB)	3.32 µs	10.7 µs (3.3x)
tag every link rel=nofollow — ars technica (56 kB)	13.2 µs	27.9 µs (2.2x)
tag every link rel=nofollow — mozilla blog (95 kB)	21 µs	41.3 µs (2.0x)
tag every link rel=nofollow — whatwg spec (235 kB)	42.5 µs	62.4 µs (1.5x)
class add/remove on every link — daring fireball (10 kB)	2.2 µs	46.4 µs (21.2x)
class add/remove on every link — ars technica (56 kB)	8.55 µs	123 µs (14.5x)
class add/remove on every link — mozilla blog (95 kB)	8.48 µs	164 µs (19.4x)
class add/remove on every link — whatwg spec (235 kB)	8.08 µs	194 µs (24.1x)
drop tags with content (remove) — daring fireball (10 kB)	25.3 µs	88.7 µs (3.6x)
drop tags with content (remove) — ars technica (56 kB)	125 µs	450 µs (3.6x)
drop tags with content (remove) — mozilla blog (95 kB)	283 µs	1.03 ms (3.7x)
drop tags with content (remove) — whatwg spec (235 kB)	687 µs	2.15 ms (3.2x)
unwrap tags keep content (strip_tags) — daring fireball (10 kB)	25.8 µs	91.7 µs (3.6x)
unwrap tags keep content (strip_tags) — ars technica (56 kB)	127 µs	467 µs (3.7x)
unwrap tags keep content (strip_tags) — mozilla blog (95 kB)	286 µs	1.04 ms (3.7x)
unwrap tags keep content (strip_tags) — whatwg spec (235 kB)	742 µs	2.14 ms (2.9x)
replace body inner HTML — daring fireball (10 kB)	2.33 µs	13.8 µs (6.0x)
replace body inner HTML — ars technica (56 kB)	9.43 µs	114 µs (12.2x)
replace body inner HTML — mozilla blog (95 kB)	15.6 µs	107 µs (6.9x)
replace body inner HTML — whatwg spec (235 kB)	45.4 µs	262 µs (5.8x)
replace body text — daring fireball (10 kB)	1.19 µs	9.24 µs (7.8x)
replace body text — ars technica (56 kB)	7.72 µs	41.9 µs (5.5x)
replace body text — mozilla blog (95 kB)	13 µs	173 µs (13.4x)
replace body text — whatwg spec (235 kB)	39.8 µs	318 µs (8.0x)
walk every descendant — daring fireball (10 kB)	3.34 µs	15.9 µs (4.8x)
walk every descendant — ars technica (56 kB)	13.8 µs	65.1 µs (4.8x)
walk every descendant — mozilla blog (95 kB)	28.2 µs	137 µs (4.9x)
walk every descendant — whatwg spec (235 kB)	101 µs	480 µs (4.8x)
extract every link — daring fireball (10 kB)	8.66 µs	145 µs (16.8x)
extract every link — ars technica (56 kB)	27.7 µs	538 µs (19.4x)
extract every link — mozilla blog (95 kB)	51.5 µs	1.17 ms (22.8x)
extract every link — whatwg spec (235 kB)	84.8 µs	3.94 ms (46.5x)
absolutize every link — daring fireball (10 kB)	82 µs	280 µs (3.5x)
absolutize every link — ars technica (56 kB)	220 µs	918 µs (4.2x)
absolutize every link — mozilla blog (95 kB)	428 µs	1.9 ms (4.5x)
absolutize every link — whatwg spec (235 kB)	282 µs	5.58 ms (19.8x)
rewrite every link — daring fireball (10 kB)	3.24 µs	163 µs (50.2x)
rewrite every link — ars technica (56 kB)	13.4 µs	605 µs (45.3x)
rewrite every link — mozilla blog (95 kB)	23.1 µs	1.38 ms (59.9x)
rewrite every link — whatwg spec (235 kB)	41.9 µs	4.4 ms (105x)
extract @href per match — daring fireball (10 kB)	2.58 µs	12.8 µs (5.0x)
extract @href per match — ars technica (56 kB)	6.65 µs	30.7 µs (4.7x)
extract @href per match — mozilla blog (95 kB)	9.23 µs	45.9 µs (5.0x)
extract @href per match — whatwg spec (235 kB)	9.81 µs	74.3 µs (7.6x)
extract text per match — daring fireball (10 kB)	2.69 µs	57 µs (21.2x)
extract text per match — ars technica (56 kB)	6.68 µs	135 µs (20.2x)
extract text per match — mozilla blog (95 kB)	10.2 µs	227 µs (22.3x)
extract text per match — whatwg spec (235 kB)	10.9 µs	295 µs (27.1x)
feed and dispatch a page — daring fireball (10 kB)	85.9 µs	77.7 µs (1.0x)
feed and dispatch a page — ars technica (56 kB)	372 µs	346 µs (1.0x)
feed and dispatch a page — mozilla blog (95 kB)	809 µs	705 µs (0.9x)
feed and dispatch a page — whatwg spec (235 kB)	2.44 ms	1.7 ms (0.7x)
css_path for every element — daring fireball (10 kB)	18 µs	103 µs (5.8x)
css_path for every element — ars technica (56 kB)	90.8 µs	545 µs (6.1x)
css_path for every element — mozilla blog (95 kB)	228 µs	1.43 ms (6.3x)
css_path for every element — whatwg spec (235 kB)	2.1 ms	5.78 ms (2.8x)
xpath_path for every element — daring fireball (10 kB)	17.8 µs	102 µs (5.8x)
xpath_path for every element — ars technica (56 kB)	92.5 µs	553 µs (6.0x)
xpath_path for every element — mozilla blog (95 kB)	262 µs	1.41 ms (5.4x)
xpath_path for every element — whatwg spec (235 kB)	2.09 ms	6.03 ms (2.9x)
XPath feature surface (9.6 kB) — //div	1.75 µs	11.5 µs (6.6x)
XPath feature surface (9.6 kB) — //a[@href]	369 ns	3.88 µs (10.6x)
XPath feature surface (9.6 kB) — //div//a[@href]	1.49 µs	10 µs (6.8x)
XPath feature surface (9.6 kB) — /html/body/div	784 ns	6.32 µs (8.1x)
XPath feature surface (9.6 kB) — //div//a[1]	9.51 µs	10 µs (1.1x)
XPath feature surface (9.6 kB) — //a[contains(@href, ‘/’)]	383 ns	4.19 µs (11.0x)
XPath feature surface (9.6 kB) — //div[position() <= 3]	5.41 µs	13.6 µs (2.6x)
XPath feature surface (9.6 kB) — //a/ancestor::div	379 ns	2.38 µs (6.3x)
XPath feature surface (9.6 kB) — //a \| //span	639 ns	3.21 µs (5.1x)
XPath feature surface (9.6 kB) — //*[local-name() = ‘a’]	4.61 µs	14.7 µs (3.2x)
XPath feature surface (9.6 kB) — count(//a)	387 ns	2.42 µs (6.3x)
XPath feature surface (9.6 kB) — //a[@href=$x] (variable)	500 ns	4.25 µs (8.6x)
XPath feature surface (9.6 kB) — //a[re:test(@href, …)] (EXSLT)	367 ns	6.2 µs (17.0x)
XPath feature surface (9.6 kB) — set:distinct(//a) (EXSLT)	420 ns	4.29 µs (10.3x)
XPath feature surface (9.6 kB) — //a/@href (smart_strings)	473 ns	2.77 µs (5.9x)
XPath feature surface (9.6 kB) — ext(//a) (extensions)	904 ns	3.41 µs (3.8x)
XPath feature surface (9.6 kB) — ext(//a)/@href (node-set extension)	952 ns	3.5 µs (3.7x)
XPath feature surface (9.6 kB) — //svg:rect (namespaces=)	567 ns	3.19 µs (5.7x)
XPath feature surface (9.6 kB) — $rows/div (node-set variable)	2.48 µs	5.35 µs (2.2x)
XPath feature surface (9.6 kB) — //a[@href] (precompiled, reused)	316 ns	3.14 µs (10.0x)

The Performance page benchmarks the full serializer, builder, editor, CSS, XPath 1.0, and EXSLT surface against lxml directly, and sweeps the node-path generators across every page size. Compiling a hot expression once with XPath (the parse happens at construction, so the call site only supplies the context node and any $name variables) stays ahead of lxml per evaluation, as the precompiled //a[@href] row shows. On the EXSLT cases, a re:test predicate runs several times ahead of lxml even though re: dispatches to Python’s re where lxml uses C libexslt, because it skips the per-call namespace resolution; lxml’s streaming evaluation narrows the node-set reductions on the multi-megabyte inputs.

How to migrate¶

The two parse entry points swap directly: turbohtml.parse() replaces lxml.html.document_fromstring and turbohtml.parse_fragment() replaces lxml.html.fromstring. The biggest change is the tree shape. lxml stores text as an element’s .text and .tail strings; turbohtml models it as real child Text nodes, so you iterate children instead of reading two string fields.

lxml	turbohtml
`el.tag`	`tag` (same)
`el.get("x")`, `el.attrib`, `el.set("x", "v")`	`attrs` (`attrs.get("x")`, `attrs["x"] = "v"`)
`el.classes.add("x")`, `el.classes.discard("x")`, `el.classes.toggle("x")`, `"x" in el.classes`	`el.add_class("x")`, `el.remove_class("x")`, `el.toggle_class("x")`, `el.has_class("x")`
`el.text`, `el.tail`	child `Text` nodes; iterate `children`
`el.text_content()`	`text`
`el.getparent()`, `el.getnext()`, `el.getprevious()`	`parent`, `next_sibling`, `previous_sibling`
`list(el)`, `el.iterdescendants()`, `el.iterancestors()`	`children`, `descendants`, `ancestors`
`el.findall(".//a")`, `el.xpath("//a[@href]")`	`find_all()`, `xpath()`
`etree.XPath("//a[@href=$u]")(el, u=v)`	`XPath` (`XPath("//a[@href=$u]")(el, u=v)`)
`el.xpath("$rows/td", rows=el.xpath("//tr"))`	`el.xpath("$rows/td", rows=el.xpath("//tr"))` (a `$name` variable binds a scalar, an `Element`, or an iterable of elements; `xpath_one()` and `xpath_iter()` take the same bindings)
`el.xpath("//svg:rect", namespaces={"svg": SVG})`	`xpath()` with the same `namespaces={"svg": SVG}` (the prefix binds at evaluation time)
`el.cssselect("div a")`	`select()`
`etree.FunctionNamespace(None)["f"] = fn`; `el.xpath("f(//a)")`	`el.xpath("f(//a)", extensions={(None, "f"): fn})` (the function may return a scalar, an `Element`, or an iterable of elements)
`el.getroottree().getpath(el)`	`el.xpath_path()` (or `el.css_path()` for a CSS selector)
`lxml.html.Element("div")`, `etree.SubElement(p, "div")`	`Element`, `p.append(Element("div"))`
`lxml.builder.E.ul(E.li("a"), E.li("b"))`	`turbohtml.build.E` (`E.<tag>(attrs, *children)` with a leading attribute mapping)
`el.drop_tag()`, `el.drop_tree()`	`unwrap()`, `decompose()`
`el.sourceline`	`source_line` (1-based, like lxml; plus `source_col`)
`el.iterlinks()`	`links()`
`el.make_links_absolute(base)`, `el.rewrite_links(fn)`	`resolve_links()`, `rewrite_links()`
`etree.iterparse(...)`	`turbohtml.IncrementalParser` (`feed` chunks, `close` for the `Document`)
`lxml.html.tostring(el)`	`html`

A query-and-select flow ports directly:

doc = parse('<div><a href="/x">go</a></div>')
print(doc.find_all("a", attrs={"href": True}))
print(doc.select_one("div a").attrs["href"])

[Element('a')]
/x

Precompile a hot XPath the same way you would reach for lxml.etree.XPath over a bare el.xpath. turbohtml’s compiled program is tree-independent, so a single object evaluates against many documents:

from turbohtml import XPath

links = XPath("//a[@href=$u]")
doc = parse('<div><a href="/x">go</a><a href="/y">stay</a></div>')
print([link.attrs["href"] for link in links(doc, u="/x")])

['/x']

The builder reads like lxml.builder.E but hands back a real Element, so the query, edit, and serialize surface stays available on what you build:

from turbohtml.build import E

print(E.ul(E.li({"class": "item"}, "one"), E.li({"class": "item"}, "two")).serialize())

<ul><li class="item">one</li><li class="item">two</li></ul>

Gotchas and pitfalls¶

No text/tail. A node’s children are its text runs and elements interleaved; read text for the concatenation.
Different tree on malformed input. lxml parses with libxml2, which is not WHATWG-conformant, so broken markup lands in a different tree than the one turbohtml (and a browser) builds. Do not expect byte-identical trees when porting scrapers that leaned on libxml2’s recovery quirks.
Custom XPath functions bind per call, not globally. lxml registers callables through etree.FunctionNamespace; turbohtml passes them through the extensions= mapping of xpath(), bound once against the compiled expression rather than a process-wide table.
Streaming differs. For a document that arrives in pieces, etree.iterparse is replaced by turbohtml.IncrementalParser: feed str or bytes chunks with feed and call close for the finished Document. It never holds the whole source at once, but it does not expose lxml’s event-driven element callbacks; you walk the completed tree.
EXSLT is built in but not exhaustive. The node-synthesizing str:tokenize/str:split and the implicit current-date date: forms stay out of scope; every other re:/set:/str:/math:/date: form ports straight through with no registration.