Performance

Every number here comes from pyperf on CPython 3.14.6 (a release build) on an Apple M4 running macOS 26. pyperf runs each case in isolated worker processes and reports the mean; the harness prints the run-to-run standard deviation beside it as ±N% so a real gap reads apart from noise, and these tables quote the mean. For the lowest-noise figures, tune the machine with pyperf system tune first (and sudo pyperf system reset after); pyperf options like --rigorous or --affinity pass straight through after the tox -e bench command. Operations that mutate the tree they are handed – the edits, the content setters, link absolutization – are timed on a fresh parse rebuilt before each iteration (the rebuild itself untimed), so the figure is the repeatable cost of the mutation alone rather than a tree a prior iteration already changed; read-path operations reuse one cached parse and time only the query. The corpora are real documents: Project Gutenberg’s War and Peace, the WHATWG HTML specification source, the ECMAScript specification, a size-weighted sample of web-platform-tests pages for the parse and tokenize suites, and, for the read-path suites, real saved web pages – a blog, a news article, and a product blog from the mozilla/readability test corpus – so the selector, link, and edit operations run against genuine nested structure rather than the layout fixtures, which carry none. The harness benchmarks each competitor in its own isolated uv venv – turbohtml in a venv of its own as the shared baseline – so one library’s dependency pins never perturb another’s. Every table below is one harness operation, so each is reproducible with tox -e bench <command>, where the command is core (turbohtml’s own baseline for every operation), an operation name (the cross-competitor table), a package name (that competitor’s own report), or all. By default the turbohtml baseline is a plain wheel, which builds quickly for iterating; pass --pgo before the command (tox -e bench -- --pgo core) to build it instead with the shipped profile-guided, link-time-optimized release recipe, so the baseline reads as what a release ships rather than a plain build, at the cost of a much slower build. Most operations are a single call; a few aggregate workloads (build, build-e) sweep a size, and the construct and emit breakdowns decompose that write path into the constructor and the serializer in isolation. Numbers vary with input and hardware.

Escaping

turbohtml.escape() against the standard library’s html.escape(). It gains the most on text that needs little escaping, where the SIMD scan classifies sixteen bytes at a time and copies clean stretches in bulk; the gap narrows on tiny strings, where call overhead dominates.

input

turbohtml

html.escape

tiny plain (64 B)

50 ns

120 ns (2.4x)

medium markup (4 KiB)

2.2 µs

7.48 µs (3.4x)

no-op prose (4 MiB)

120 µs

2.56 ms (21.4x)

book text (3 MiB)

670 µs

2.65 ms (4.0x)

book HTML (4 MiB)

1.25 ms

4.65 ms (3.8x)

spec HTML, dense (4 MiB)

4.94 ms

13 ms (2.7x)

UCS-2 plain (4 MiB)

800 µs

2.46 ms (3.1x)

UCS-2 markup (4 MiB)

5.6 ms

11 ms (2.0x)

UCS-4 plain (4 MiB)

920 µs

5.29 ms (5.8x)

UCS-4 markup (4 MiB)

7.2 ms

20 ms (2.8x)

Markup (escaping)

turbohtml.migration.markupsafe.escape() against markupsafe’s own C escape, both returning a Markup. The inputs are the small, mostly-clean strings a template engine interpolates under autoescape, markupsafe’s hottest path. turbohtml builds the safe string in C in a single call, where markupsafe pays a Python escape frame and Markup construction per call, so it runs roughly two to four times faster.

input

turbohtml

markupsafe

clean (8 B)

60 ns

186 ns (3.1x)

clean (32 B)

70 ns

204 ns (3.0x)

clean (256 B)

124 ns

449 ns (3.7x)

name with ' and &

83 ns

210 ns (2.6x)

escape-heavy markup

138 ns

339 ns (2.5x)

The other Markup operations race markupsafe’s own Markup of the same method. striptags and unescape run on turbohtml’s tokenizer and HTML5 reference resolution where markupsafe scans with a regex, and format and join escape each untrusted operand through the same C escape.

operation

turbohtml

markupsafe

striptags

1.13 µs

2.26 µs (2.0x)

unescape

211 ns

965 ns (4.6x)

format (escapes operands)

1.8 µs

2.01 µs (1.2x)

join (escapes operands)

640 ns

1.19 µs (1.9x)

Linkify

turbohtml.clean.linkify() against bleach’s linkify, the HTML-aware linkifier it succeeds, and linkify-it-py, the pure-Python scanner markdown-it-py pulls in. bleach and turbohtml both parse the HTML and rewrite it; linkify-it-py only finds the matches and does not rewrite, so it does strictly less work, yet turbohtml is faster than both. The C candidate scan and turbohtml’s own tree carry it past bleach’s html5lib pass by five to eighteen times.

input

turbohtml

lxml-html-clean

bleach

comment (1 link, 1 email)

3.05 µs

6.45 µs (2.2x)

54.3 µs (17.9x)

prose (1 KiB)

49.4 µs

22.5 µs (0.5x)

269 µs (5.5x)

markup (4 KiB)

121 µs

138 µs (1.2x)

1.6 ms (13.3x)

The detection primitive on its own, turbohtml.clean.LinkDetector.find() against LinkifyIt().match and has_link() against LinkifyIt().test, scans a run of plain text and returns the spans or a boolean without rewriting any HTML, so this isolates the C scan from the full linkify rewrite above. The has_link prose row is close because test short-circuits on the first link near the start.

detect

turbohtml

linkify-it-py

find comment (1 link, 1 email)

600 ns

30 µs (50.0x)

find prose (1 KiB)

8.4 µs

320 µs (38.1x)

has_link comment

300 ns

21.5 µs (71.7x)

has_link prose (1 KiB)

2.4 µs

4.8 µs (2.0x)

Sanitize

turbohtml.clean.sanitize() against four sanitizers. Three share its allowlist model, where only listed tags and attributes survive, so a vector nobody anticipated is dropped by default: nh3 (the Rust ammonia binding), bleach (its end-of-life predecessor, on html5lib), and html-sanitizer (an allowlist over lxml). The fourth, lxml-html-clean (the externalized lxml.html.clean.Cleaner), is a blocklist: it strips the constructs it knows are dangerous and lets the rest through, a model lxml itself flagged as hard to keep safe. The inputs are realistic user content with a few disallowed tags and a dangerous attribute mixed in. turbohtml runs the whole filtering walk in C and leads every alternative, but the model matters more than the microseconds. Prefer an allowlist, since a blocklist passes anything it did not think to name.

input

turbohtml

nh3

lxml-html-clean

html-sanitizer

bleach

comment (1 link, 1 script)

1.4 µs

5.7 µs (4.1x)

20 µs (14.3x)

46.1 µs (33.0x)

78 µs (55.8x)

post (4 KiB)

39.5 µs

144 µs (3.7x)

520 µs (13.2x)

1.62 ms (41.0x)

1.99 ms (50.4x)

Template-safe sanitizing (Policy.strip_template_markers, collapsing {{ }}/${ }/<% %> so the output cannot re-inject through a template engine) has no allowlist-sanitizer analog in Python; the reference is DOMPurify’s SAFE_FOR_TEMPLATES, which runs in JavaScript. Reaching it from Python means shelling out to Node, where each call spins up a DOM before it sanitizes, so the figure below is that end-to-end per-document cost, not a pure-algorithm comparison. turbohtml folds the same transform into its C walk and pays neither the process nor the DOM.

input

turbohtml

DOMPurify

templated 4 KiB

142 µs

259 ms (1826x)

Markdown

turbohtml.Node.to_markdown() against markdownify (on BeautifulSoup) and html2text (a streaming HTMLParser subclass). All three take an HTML string and return Markdown, so each parses first; turbohtml parses to the WHATWG tree and walks it in C, where the others build and convert in Python. The single C pass converts a page in tens of microseconds, two orders of magnitude ahead of markdownify. The configured row turns the option surface on in all three (underscore emphasis, reference links, padded tables, full escaping), and turbohtml stays ahead by the same margin.

input

turbohtml

html2text

markdownify

article (2 KiB)

2.6 µs

530 µs (204x)

1.2 ms (462x)

list (4 KiB)

4 µs

1.16 ms (289x)

2.36 ms (592x)

table (4 KiB)

6.4 µs

1.01 ms (158x)

2.84 ms (444x)

configured (4 KiB)

13 µs

1.11 ms (85.1x)

2.59 ms (200x)

google_doc (4 KiB)

9.9 µs

600 µs (60.7x)

—1

1 no equivalent operation

The google_doc row reads the inline-CSS styling a Google Docs export carries (html2text’s google_doc mode); markdownify has no equivalent.

Structured data

turbohtml.Document.structured_data() against extruct, the scraper toolkit it succeeds, extracting JSON-LD, Microdata, and OpenGraph from a product page that carries all three. Both start from the raw HTML string, so each parses first; extruct builds an lxml tree and runs a separate extractor per syntax, where turbohtml parses to the WHATWG tree and gathers every format in one C walk, handing back the typed StructuredData record. The single pass runs roughly nine to eleven times faster.

input

turbohtml

extruct

product

5.37 µs

58.3 µs (10.9x)

catalog 8 KiB

54 µs

504 µs (9.4x)

Tables

turbohtml.Node.tables() and turbohtml.Element.records() against pandas’s read_html, the one-call table reader scrapers reach for. Both parse the HTML and extract every <table>, resolving rowspan and colspan into a rectangular grid; read_html returns a DataFrame per table and pulls in NumPy, where turbohtml runs the cell-grid walk in C and hands back plain list and dict objects with no added dependency. The rows row times tables() (every table as list[list[str]]) and the records row times records() (the first table keyed by its header), each over a four-column table of the given height. The single C pass leads from roughly twenty times on the largest table to over a hundred and thirty on the smallest, where pandas pays its fixed per-frame construction cost.

input

turbohtml

pandas

rows (10 rows)

2.6 µs

345 µs (133x)

records (10 rows)

2.84 µs

440 µs (155x)

rows (100 rows)

23.5 µs

1.01 ms (42.9x)

records (100 rows)

24.5 µs

1.14 ms (46.6x)

rows (1000 rows)

259 µs

7.52 ms (29.1x)

records (1000 rows)

389 µs

8.2 ms (21.1x)

Article extraction

turbohtml.Node.article() against trafilatura, readability-lxml, newspaper3k, goose3, readabilipy, and news-please, the article extractors it succeeds. Each scores the dominant content body and (trafilatura, newspaper3k, goose3, and news-please) harvests the page metadata beside it; the lxml-backed four build their tree in Python first, readabilipy’s Python mode parses with html5lib into BeautifulSoup and cleans without scoring, news-please merges the votes of several such extractors, and turbohtml does the scoring and the harvest in one C pass over the parsed tree. The inputs are full pages – navigation, a scored article, and a footer – so the boilerplate the heuristic discounts is part of the measured cost.

input

turbohtml

readability-lxml

trafilatura

newspaper3k

goose3

news-please

readabilipy

post (4 KiB)

7.1 µs

500 µs (70.5x)

600 µs (84.6x)

1.91 ms (270x)

2.15 ms (303x)

5.2 ms (733x)

5 ms (705x)

longform (16 KiB)

28 µs

1.52 ms (54.3x)

1.7 ms (60.8x)

5.5 ms (197x)

7.75 ms (277x)

14.4 ms (515x)

16.6 ms (593x)

Boilerplate classification

turbohtml.extract.boilerplate() against justext and boilerpy3, the per-block boilerplate classifiers. All three segment the page into units and mark each good or boilerplate; justext scores every paragraph in Python over an lxml tree (length, link density, stopword density), boilerpy3 classifies the blocks of its own SAX stream with boilerpipe’s rules, and turbohtml scores the tree once in C and classifies the units in a thin Python layer. The inputs are the article-extraction pages, so the navigation and footer each classifier must reject are part of the measured cost.

input

turbohtml

boilerpy3

justext

post (4 KiB)

24.7 µs

249 µs (10.1x)

803 µs (32.6x)

longform (16 KiB)

90 µs

870 µs (9.7x)

3.29 ms (36.6x)

Date extraction

turbohtml.extract.dates() against htmldate, the standalone publication-date finder. Both read the same signals – publication/modification <meta> tags, JSON-LD, <time> elements, and a date in the URL – and both are parse-bound; htmldate builds an lxml tree, turbohtml the WHATWG tree. turbohtml’s early-exit over the structured signals runs two to three times faster on the pages below.

publication-date extraction

turbohtml

htmldate

post (4 KiB)

8.54 µs

17.5 µs (2.1x)

longform (16 KiB)

12.5 µs

33.2 µs (2.7x)

Unescaping

turbohtml.unescape() against html.unescape() and w3lib’s replace_entities, the Scrapy helper that resolves the same references. It gains the most on entity-heavy input, where the standard library pays a Python call per match and w3lib runs a regular-expression substitution with a Python callback per match; turbohtml hops between & occurrences in C and bulk-copies the clean spans between references.

input

turbohtml

html.unescape

w3lib

tiny plain (64 B)

30 ns

40 ns (1.4x)

250 ns (8.4x)

medium dense refs (4 KiB)

7 µs

71.4 µs (10.3x)

114 µs (16.3x)

numeric refs (4 KiB)

4.84 µs

79.1 µs (16.4x)

91.9 µs (19.0x)

book HTML, real refs (4 MiB)

3.29 ms

8.32 ms (2.6x)

14.6 ms (4.5x)

escaped book HTML (5 MiB)

1.61 ms

20 ms (12.5x)

37 ms (23.0x)

dense refs (4 MiB)

8.29 ms

73.6 ms (8.9x)

120 ms (14.5x)

UCS-2 refs (4 MiB)

2.47 ms

18.6 ms (7.6x)

28.1 ms (11.4x)

Tokenizing

turbohtml.tokenize() against html.parser.HTMLParser (driven with no-op handlers) and html5lib’s pure-Python tokenizer. The closest case is a document dominated by a single text node, where the standard library’s regex performs one C scan; wherever markup appears, the state machine is roughly ten to fifteen times faster.

input

turbohtml

html.parser

html5lib

typical markup

29 µs

437 µs (15.1x)

811 µs (28.0x)

text-heavy prose

540 ns

2.8 µs (5.2x)

146 µs (271x)

attribute-heavy

18.2 µs

300 µs (16.5x)

797 µs (43.8x)

script-heavy

11.2 µs

153 µs (13.7x)

500 µs (44.7x)

entity-heavy

20 µs

191 µs (9.6x)

1.2 ms (60.0x)

wpt page (0.6 kB)

1.5 µs

17.2 µs (11.5x)

47.9 µs (32.0x)

wpt page (4 kB)

12 µs

168 µs (14.0x)

426 µs (35.5x)

wpt page (9.6 kB)

27.5 µs

360 µs (13.1x)

1.16 ms (42.2x)

wpt page (92 kB)

322 µs

4 ms (12.5x)

9.24 ms (28.7x)

wpt page, CJK (124 kB)

550 µs

8.49 ms (15.5x)

22.4 ms (40.8x)

whatwg spec (235 kB)

767 µs

7.6 ms (10.0x)

19.3 ms (25.2x)

ecmascript spec (3 MB)

6.2 ms

54 ms (8.8x)

180 ms (29.1x)

whatwg spec source (7.9 MB)

36.3 ms

380 ms (10.5x)

850 ms (23.5x)

Parsing

turbohtml.parse() builds a full WHATWG document tree, against the other Python tree builders: lxml, selectolax and resiliparse (both wrapping lexbor), BeautifulSoup over html.parser, and html5lib. turbohtml runs on par with resiliparse, three to six times faster than lxml and selectolax, and 35 to 85 times faster than the pure-Python builders, while building the WHATWG tree that lxml’s libxml2 does not.

resiliparse reaches turbohtml’s throughput because its HTMLTree.parse is a thin call straight into lexbor’s native tree, while selectolax wraps that same engine behind a heavier object layer; the comparison here is parsing only. resiliparse’s wider toolkit, boilerplate and main-content extraction, language detection, and the encoding and archive utilities it ships for large-scale web-crawl processing, sits outside turbohtml’s scope. gumbo, the C WHATWG parser Google released and the engine behind html5-parser, has no row: it is read-oriented, archived upstream, and its Python binding no longer builds on a current toolchain. turbohtml is the maintained, mutable, typed alternative to that lineage.

input

turbohtml

resiliparse

lxml

selectolax

BeautifulSoup

html5lib

wpt page (0.6 kB)

1.2 µs

3.8 µs (3.2x)

3.4 µs (2.9x)

6.9 µs (5.8x)

60.6 µs (50.6x)

99.6 µs (83.0x)

wpt page (4 kB)

9.58 µs

12.8 µs (1.4x)

28 µs (3.0x)

42.2 µs (4.5x)

450 µs (47.0x)

610 µs (63.7x)

wpt page (9.6 kB)

24.1 µs

28.4 µs (1.2x)

75 µs (3.2x)

107 µs (4.5x)

840 µs (34.9x)

1.43 ms (59.4x)

wpt page (92 kB)

209 µs

282 µs (1.4x)

645 µs (3.1x)

922 µs (4.5x)

15 ms (71.8x)

17 ms (81.4x)

wpt page, CJK (124 kB)

410 µs

549 µs (1.4x)

1.5 ms (3.7x)

2.3 ms (5.7x)

23.3 ms (56.9x)

29.2 ms (71.3x)

whatwg spec (235 kB)

400 µs

504 µs (1.3x)

1.26 ms (3.2x)

1.79 ms (4.5x)

25 ms (62.5x)

32.2 ms (80.5x)

Fragment parsing

turbohtml.parse_fragment() parses an innerHTML-style snippet in a container’s context rather than a whole document, against lxml’s lxml.html.fromstring and html5lib’s parseFragment. The input is a table-row fragment parsed in its <tbody> context, where the WHATWG algorithm’s table rules apply. turbohtml runs the same C engine it uses for whole documents, so it parses the fragment three times faster than lxml and roughly seventy-five times faster than the pure-Python html5lib.

input

turbohtml

lxml

html5lib

table-row fragment (2 kB)

11 µs

34.5 µs (3.2x)

837 µs (76.1x)

Querying

Each library parses the document once, then the timed call runs one query. find collects every <a> element the way each library reaches for it (turbohtml’s find_all(), lxml’s XPath findall, selectolax’s and BeautifulSoup’s selectors). A tag-only query resolves the name to an interned atom and walks the subtree comparing integers, with no per-element string built and no matcher dispatch, so it runs ahead of lxml’s C XPath engine and many times ahead of selectolax, parsel (Scrapy’s cssselect-over-libxml2 selector library), and BeautifulSoup.

find every <a>

turbohtml

lxml

selectolax

parsel

BeautifulSoup

daring fireball (10 kB)

400 ns

5 µs (12.6x)

5.6 µs (14.0x)

20.7 µs (51.8x)

13.5 µs (33.8x)

ars technica (56 kB)

800 ns

13.4 µs (16.8x)

15.8 µs (19.8x)

47.3 µs (59.2x)

47.9 µs (59.9x)

mozilla blog (95 kB)

1.2 µs

21.1 µs (17.6x)

28.5 µs (23.8x)

67.6 µs (56.4x)

109 µs (90.9x)

whatwg spec (235 kB)

1.4 µs

35.6 µs (25.5x)

76 µs (54.3x)

104 µs (74.5x)

350 µs (251x)

select runs the CSS selector div a[href] (turbohtml’s select(), lxml’s cssselect, selectolax’s css, parsel’s css, BeautifulSoup’s soupsieve). Because turbohtml compiles the selector against the tree once and then matches by comparing interned integer atoms, it stays in the low microseconds across these pages. lxml and parsel re-translate the selector to XPath through cssselect on every call, which scales with the document and trails by tens of times on the small blog up to roughly seven hundred fifty times on the spec; BeautifulSoup’s soupsieve is hundreds to more than fifteen hundred times behind, while selectolax, the other compiled engine, stays closest at roughly twelve to forty-five times.

select div a[href]

turbohtml

selectolax

lxml

parsel

BeautifulSoup

daring fireball (10 kB)

600 ns

7.5 µs (12.5x)

29.4 µs (49.0x)

30 µs (50.0x)

169 µs (282x)

ars technica (56 kB)

1.5 µs

19.9 µs (13.3x)

126 µs (84.0x)

143 µs (95.6x)

570 µs (381x)

mozilla blog (95 kB)

2.1 µs

34 µs (16.2x)

800 µs (382x)

840 µs (401x)

1.1 ms (524x)

whatwg spec (235 kB)

1.8 µs

79.4 µs (44.2x)

1.35 ms (751x)

1.37 ms (762x)

2.79 ms (1550x)

The relational :has() pseudo-class is the costliest selector to evaluate, since a naive matcher rescans each candidate’s subtree. turbohtml runs div:has(a) against the same pages and leads every alternative: tens of times faster than lxml and selectolax on the smaller pages, narrowing to single digits on the link-dense mozilla blog where the relational match itself does real work, while BeautifulSoup trails by hundreds of times throughout. The matcher walks each anchor’s descendants once and skips the sibling scan for descendant and child relationships, so the relational lookup keeps the same interned-atom comparison the flat selectors use.

select div:has(a)

turbohtml

selectolax

lxml

BeautifulSoup

daring fireball (10 kB)

300 ns

5 µs (16.7x)

14.3 µs (47.7x)

114 µs (379x)

ars technica (56 kB)

1.2 µs

18 µs (15.1x)

28.2 µs (23.6x)

490 µs (409x)

mozilla blog (95 kB)

9 µs

50.9 µs (5.7x)

60.7 µs (6.8x)

2.2 ms (245x)

whatwg spec (235 kB)

5.8 µs

83.3 µs (14.4x)

71 µs (12.3x)

3.2 ms (552x)

Per-element matching runs each anchor on the page through a compiled div a[href] matcher – the shape a soupsieve port hits through turbohtml.query and its Matcher.match (soupsieve is the only competitor with a compiled per-element match). turbohtml answers each test with the same interned-atom comparison its select uses, walking the ancestor chain once per candidate, where soupsieve re-interprets the parsed selector in Python per element, so the sweep runs 78 to 148 times faster across these pages.

match each anchor against div a[href]

turbohtml

soupsieve

daring fireball (10 kB)

1.79 µs

140 µs (78.3x)

ars technica (56 kB)

4.21 µs

401 µs (95.3x)

mozilla blog (95 kB)

6 µs

609 µs (102x)

whatwg spec (235 kB)

6.7 µs

992 µs (149x)

A text-content search runs through find_all() with text= (a regex matched against each element’s collected subtree text), raced against BeautifulSoup.find_all(string=...); lxml, selectolax, and parsel expose no equivalent, so this is a two-way race. When the text= filter is a plain string or a literal (no regex metacharacters, case-sensitive) compiled pattern, turbohtml gathers each candidate’s collected text and matches it in C – no Python str built, no per-element re.search call – where BeautifulSoup’s find_all(string=...) walks the tree in Python, so turbohtml now leads on every page (it previously trailed on the larger ones). A case-insensitive or otherwise non-literal pattern keeps the per-element Python path.

find text= regex

turbohtml

BeautifulSoup

daring fireball (10 kB)

24 µs

57.7 µs (2.5x)

ars technica (56 kB)

178 µs

217 µs (1.3x)

mozilla blog (95 kB)

287 µs

463 µs (1.7x)

whatwg spec (235 kB)

560 µs

1.67 ms (3.0x)

turbohtml.convert.css_specificity() weighs a selector list’s (a, b, c) specificity, raced against cssselect’s Selector.specificity(), the computation lxml, parsel, and pyquery inherit. turbohtml parses the selector and sums the weights in one C pass, so it leads across the type, compound, structural, complex, and grouped selectors below; cssselect parses in Python and builds a tree of selector objects first.

selector

turbohtml

cssselect

type

325 ns

665 ns (2.1x)

compound

1.12 µs

17.7 µs (15.8x)

structural

997 ns

14.6 µs (14.7x)

complex

1.78 µs

31.4 µs (17.7x)

group

1.33 µs

20 µs (15.0x)

XPath 1.0 evaluation runs through xpath(), raced against lxml’s libxml2 engine, the XPath that parsel, pyquery, and html5-parser all wrap (selectolax and BeautifulSoup have none). One expression per feature class (name tests, the // abbreviation, attribute, positional, and arithmetic predicates, string and aggregate functions, a reverse axis, a union, and a computed name test) runs over the 9.6 kB wpt page below; tox -e bench xpath repeats the sweep across every page size. turbohtml compiles each expression against the tree once, resolves name tests to interned atoms, and folds // to a single descendant walk, so it leads across the surface. The exception is a predicate that references position() ([1] or position() <= 3): it pins the result to proximity order and disables the // collapse, so on the largest pages lxml’s streaming evaluation closes the gap. The last eight rows are the lxml/parsel options the parity work added: a $variable binding, an EXSLT re:test predicate (turbohtml’s Python re against lxml’s C libexslt), an EXSLT set:distinct node-set reduction (built-in C dispatch on both sides, so it races C against C), a smart_strings attribute read, a custom extensions= function, an extensions= function whose return becomes a node-set feeding a later /@href step, a namespaces= prefix binding that resolves //svg:rect against {"svg": ".../2000/svg"} over a page carrying an SVG block, and a node-set $variable bound from a prior result ($rows/div, with rows reused from an earlier //div query) fed into a later path step. turbohtml still leads, since lxml resolves the namespace map and option set on every call. The last row precompiles the expression once with XPath and re-evaluates it, lxml’s etree.XPath doing the same: both skip the per-call parse xpath() pays, and turbohtml’s compiled program stays ahead per evaluation.

xpath (9.6 kB page)

turbohtml

lxml

//div

1.8 µs

11.3 µs (6.3x)

//a[@href]

400 ns

3.9 µs (9.8x)

//div//a[@href]

1 µs

10 µs (10.1x)

/html/body/div

800 ns

6.1 µs (7.7x)

//div//a[1]

10.2 µs

10 µs (1.0x)

//a[contains(@href, '/')]

400 ns

4.1 µs (10.3x)

//div[position() <= 3]

5.5 µs

13.4 µs (2.5x)

//a/ancestor::div

400 ns

2.4 µs (6.0x)

//a | //span

600 ns

3.2 µs (5.4x)

//*[local-name() = 'a']

4 µs

16 µs (4.0x)

count(//a)

400 ns

2.4 µs (6.0x)

//a[@href=$x] (variable)

500 ns

4.1 µs (8.2x)

//a[re:test(@href, ...)] (EXSLT)

400 ns

7.1 µs (17.8x)

set:distinct(//a) (EXSLT)

400 ns

4 µs (10.0x)

//a/@href (smart_strings)

500 ns

2.6 µs (5.2x)

ext(//a) (extensions)

900 ns

3 µs (3.4x)

ext(//a)/@href (node-set extension)

900 ns

3 µs (3.4x)

//svg:rect (namespaces=)

600 ns

2.9 µs (4.9x)

$rows/div (node-set variable)

2.5 µs

4.6 µs (1.9x)

//a[@href] (precompiled, reused)

300 ns

2.7 µs (9.0x)

Node paths

turbohtml.Element.css_path() and xpath_path() return the unique locator that re-finds an element from the document root – a CSS selector and a positional XPath – against lxml’s getroottree().getpath(), the libxml2 path builder devtools’ “copy selector” mirrors. lxml emits only the positional XPath, so getpath pairs with both turbohtml methods. Each timed call walks every element in a pre-parsed page and serializes its path. Both methods lead getpath by roughly five to six times across these pages, narrowing to under threefold on the spec. css_path() previously rescanned the whole document to test each element’s id uniqueness, an O(N2) cost over a page that made it slower than getpath on id-heavy pages; a cached per-tree id-occurrence map (dropped with the element index on any mutation) now answers that test in O(1), so css_path keeps pace with the positional xpath_path. The ratio on getpath is against xpath_path, the like-for-like locator.

input

turbohtml css_path

turbohtml xpath_path

lxml getpath

daring fireball (10 kB)

18.9 µs

18.9 µs

104 µs (5.5x)

ars technica (56 kB)

91.4 µs

98.8 µs

557 µs (6.1x)

mozilla blog (95 kB)

227 µs

277 µs

1.43 ms (6.3x)

whatwg spec (235 kB)

2.24 ms

2.2 ms

6.11 ms (2.8x)

Text content

The text suite collects the visible text two ways. First, the raw text join off a pre-parsed tree, the get_text pass: turbohtml’s text property concatenates every descendant text run, against lxml’s text_content(), selectolax’s text(), and BeautifulSoup’s get_text(). turbohtml gathers the runs in one C walk into a buffer reserved up front, so it leads lxml by a small margin and selectolax and BeautifulSoup by roughly an order of magnitude. parsel exposes no node-level text collector, so it sits out.

text content

turbohtml

lxml

BeautifulSoup

selectolax

daring fireball (10 kB)

2.7 µs

3.3 µs (1.3x)

19 µs (7.1x)

21.5 µs (8.0x)

ars technica (56 kB)

13 µs

15 µs (1.2x)

77.6 µs (6.0x)

91 µs (7.1x)

mozilla blog (95 kB)

22.4 µs

25.4 µs (1.2x)

162 µs (7.3x)

204 µs (9.1x)

whatwg spec (235 kB)

75.4 µs

89.1 µs (1.2x)

566 µs (7.6x)

753 µs (10.0x)

Second, the layout-aware string-to-text extraction: turbohtml.Node.to_text() against inscriptis, the layout-aware HTML-to-text renderer it succeeds, html-text, Zyte’s plainer visible-text extractor, and resiliparse’s extract_plain_text. inscriptis and html-text both build an lxml tree in Python and resiliparse renders text off the lexbor tree it parses to, where turbohtml does the whole layout in one C walk; inscriptis additionally lays tables out as aligned columns, which html-text and resiliparse skip.

input

turbohtml

resiliparse

html-text

inscriptis

article (2 KiB)

2 µs

20 µs (10.1x)

92.8 µs (46.5x)

170 µs (85.1x)

table (4 KiB)

10 µs

51 µs (5.1x)

250 µs (25.0x)

900 µs (90.0x)

collapsed (2 KiB)

2 µs

—1

95 µs (47.6x)

—1

main (4 KiB)

3 µs

20 µs (6.7x)

—1

—1

annotated (4 KiB)

3 µs

—1

—1

201 µs (67.0x)

1 no equivalent operation

The collapsed row turns layout guessing off: turbohtml joins the stripped_strings word stream against html-text’s extract_text(guess_layout=False); inscriptis and resiliparse have no comparable collapsed mode. The main row strips page boilerplate first, main_text() against resiliparse’s extract_plain_text(main_content=True). The annotated row labels matching elements with spans through to_annotated_text() against inscriptis’s get_annotated_text; html-text and resiliparse have no annotation surface, so they sit out that row.

Tree navigation

Walking every descendant of a parsed tree: turbohtml’s descendants iterator against lxml’s iterdescendants() and BeautifulSoup’s descendants. The list(el), iterdescendants(), and iterancestors() family ports to children, descendants, and ancestors; the descendant walk is the dominant case. Each timed call consumes the whole iterator, where turbohtml yields interned nodes straight from the arena faster than lxml’s libxml2 proxy objects and BeautifulSoup’s Python NavigableString chain. The descendant walk is one of BeautifulSoup’s leaner paths, so the margin is narrower here than on the query and serialize suites. selectolax exposes no document-wide descendant iterator, so it has no entry.

descendant walk

turbohtml

BeautifulSoup

lxml

daring fireball (10 kB)

3.3 µs

7 µs (2.2x)

16 µs (4.9x)

ars technica (56 kB)

13 µs

26 µs (2.0x)

61.4 µs (4.8x)

mozilla blog (95 kB)

28 µs

58 µs (2.1x)

136 µs (4.9x)

whatwg spec (235 kB)

96.8 µs

190 µs (2.0x)

468 µs (4.9x)

Serializing

Serializing a parsed document back to HTML: turbohtml’s html, lxml’s tostring, selectolax’s html, and BeautifulSoup’s decode. turbohtml scans each text run for the next character that needs escaping (two code points at a time with the same SWAR lane probes escape() uses) and bulk-copies the clean spans, recovering each special’s position from the lane mask, and reserves the whole-document buffer up front so the output grows in one allocation. It serializes four to six times faster than lxml, about four times faster than selectolax, and forty-five to sixty times faster than BeautifulSoup.

serialize to HTML

turbohtml

selectolax

lxml

BeautifulSoup

daring fireball (10 kB)

7 µs

28.4 µs (4.1x)

38.5 µs (5.5x)

414 µs (59.2x)

ars technica (56 kB)

38.8 µs

154 µs (4.0x)

195 µs (5.1x)

1.8 ms (46.4x)

mozilla blog (95 kB)

76.3 µs

307 µs (4.1x)

425 µs (5.6x)

4.12 ms (54.0x)

whatwg spec (235 kB)

195 µs

740 µs (3.8x)

805 µs (4.2x)

11 ms (56.4x)

Minifying

Minifying a document with turbohtml.clean.minify(): parse, then serialize once with every fold engaged (collapsing insignificant whitespace, omitting the WHATWG-optional tags, unquoting attributes, and stripping comments), against minify-html’s Rust minifier on the same folds (its CSS and JS minification left off for a like-for-like comparison), the pure-Python htmlmin and css-html-js-minify, and the native CLI minifiers html-minifier-terser and tdewolff/minify. turbohtml parses and emits in C through one preallocated buffer, so with the parse included it runs roughly two to three times faster than minify-html and sixteen to seventy times faster than the pure-Python pair. html-minifier-terser and tdewolff are invoked through their command line, so their millisecond timings are dominated by process startup; the clean comparison against them is output size. turbohtml lands within about two percent of html-minifier-terser on the structural folds both apply; minify-html folds more aggressively for roughly three to ten percent smaller, and tdewolff goes further still by also minifying the inline CSS and JavaScript turbohtml leaves untouched here.

minify a document

turbohtml

tdewolff

minify-html

css-html-js-minify

htmlmin

html-minifier-terser

size

time

size

time

size

time

size

time

size

time

size

time

daring fireball (10 kB)

8.5 kB

27.7 µs

6.4 kB (0.76x)

3.76 ms (137x)

7.9 kB (0.94x)

69 µs (2.5x)

7.4 kB (0.87x)

631 µs (22.9x)

8.6 kB (1.01x)

597 µs (21.6x)

8.3 kB (0.98x)

73.4 ms (2654x)

ars technica (56 kB)

38.7 kB

125 µs

—1

—1

37.0 kB (0.95x)

318 µs (2.6x)

38.5 kB (0.99x)

5.45 ms (43.7x)

38.2 kB (0.99x)

2.69 ms (21.6x)

38.0 kB (0.98x)

78.6 ms (630x)

mozilla blog (95 kB)

54.2 kB

263 µs

48.7 kB (0.90x)

4.47 ms (17.0x)

48.9 kB (0.90x)

736 µs (2.8x)

53.3 kB (0.98x)

17.8 ms (67.6x)

54.4 kB (1.003x)

6.11 ms (23.3x)

53.6 kB (0.99x)

83.8 ms (319x)

whatwg spec (235 kB)

209.1 kB

918 µs

199.4 kB (0.95x)

6.13 ms (6.7x)

203.0 kB (0.97x)

1.7 ms (1.9x)

227.1 kB (1.09x)

37.2 ms (40.5x)

209.6 kB (1.003x)

15.1 ms (16.5x)

210.3 kB (1.01x)

91.4 ms (99.7x)

1 cannot minify -: unexpected < in expression on line 109 and column 17

Building

The write path: construct a <ul> of N <li> rows from scratch (each with a class, a data attribute, and a text child), then serialize it, the work an editor or template engine does. turbohtml’s arena allocation and interned attribute names make construction cheaper than lxml’s libxml2 nodes and far cheaper than BeautifulSoup’s Python objects. selectolax is parse-only, so it has no entry.

build a list

turbohtml

lxml

BeautifulSoup

100 rows

56.8 µs

130 µs (2.3x)

756 µs (13.4x)

1000 rows

540 µs

1.32 ms (2.5x)

7.4 ms (13.8x)

10000 rows

5.34 ms

13.2 ms (2.5x)

81 ms (15.2x)

The construct and emit commands split that aggregate into its two halves over the same tree: construct builds the rows and stops before serialization, and emit serializes a tree built once outside the timed region. The split shows where each library spends the time – turbohtml’s arena keeps construction roughly twice as fast as lxml, and its SWAR serializer pulls emit ahead by roughly eight times, while BeautifulSoup pays its Python object cost on both halves.

construct (no serialize)

turbohtml

lxml

BeautifulSoup

100 rows

45 µs

90 µs (2.0x)

262 µs (5.9x)

1000 rows

482 µs

952 µs (2.0x)

2.55 ms (5.3x)

10000 rows

4.42 ms

9.4 ms (2.2x)

25.3 ms (5.8x)

emit a built tree

turbohtml

lxml

BeautifulSoup

100 rows

4.2 µs

33.7 µs (8.1x)

389 µs (92.7x)

1000 rows

42 µs

320 µs (7.7x)

4 ms (95.3x)

10000 rows

420 µs

3.3 ms (7.9x)

39.1 ms (93.1x)

The terse turbohtml.build.E builder spells the same <ul> declaratively, raced against the dedicated HTML generators dominate and yattag. E is three to four times faster than dominate and on par with yattag across the sweep, and unlike either it returns a real, queryable turbohtml tree rather than a string. That tree costs a little over twice the raw Element constructor above – the price of the leading-mapping and per-child dispatch the sugar runs in Python.

build with E

turbohtml

yattag

dominate

100 rows

117 µs

147 µs (1.3x)

438 µs (3.8x)

1000 rows

1.2 ms

1.36 ms (1.2x)

4.42 ms (3.7x)

10000 rows

14.3 ms

13.7 ms (1.0x)

47 ms (3.3x)

Editing

Editing a parsed tree: tag every <a> with rel="nofollow", a link-rewriting pass. Because the pass mutates the tree, each library rebuilds a fresh parse before every iteration outside the timed region, then the timed call walks its links and sets the attribute (turbohtml through the live attrs mapping, lxml through Element.set, BeautifulSoup through item assignment). selectolax mutation is limited, so it has no entry.

tag every link

turbohtml

lxml

daring fireball (10 kB)

3 µs

10.8 µs (3.6x)

ars technica (56 kB)

13 µs

27.6 µs (2.2x)

mozilla blog (95 kB)

20 µs

41.1 µs (2.1x)

whatwg spec (235 kB)

43 µs

62 µs (1.5x)

A second pass churns the class list: add then drop a token on every link (turbohtml’s add_class()/remove_class() against lxml’s classes set). The add-then-remove is a net no-op, so each repeat does equal work. BeautifulSoup has no class-token mutator, so it has no entry.

class add/remove on every link

turbohtml

lxml

daring fireball (10 kB)

2.2 µs

47 µs (21.4x)

ars technica (56 kB)

8.9 µs

130 µs (14.7x)

mozilla blog (95 kB)

8.78 µs

160 µs (18.3x)

whatwg spec (235 kB)

8.7 µs

196 µs (22.6x)

Two content setters replace the body’s children on a freshly parsed tree. set_inner_html() reparses a fixed fragment in the <body>’s context and splices it in one C call, against lxml clearing the body and appending fragments_fromstring, BeautifulSoup clearing it and appending a reparsed soup, and pyquery’s .html(). set_text() replaces the children with one verbatim text node, against pyquery’s .text().

replace body inner HTML

turbohtml

lxml

pyquery

BeautifulSoup

daring fireball (10 kB)

2.1 µs

13.8 µs (6.6x)

18 µs (8.6x)

85.9 µs (41.0x)

ars technica (56 kB)

8.69 µs

47.6 µs (5.5x)

13.3 µs (1.6x)

132 µs (15.2x)

mozilla blog (95 kB)

14.4 µs

105 µs (7.3x)

123 µs (8.6x)

211 µs (14.7x)

whatwg spec (235 kB)

40.1 µs

260 µs (6.5x)

370 µs (9.3x)

789 µs (19.7x)

replace body text

turbohtml

pyquery

daring fireball (10 kB)

1.2 µs

15.4 µs (12.9x)

ars technica (56 kB)

7.7 µs

13.9 µs (1.9x)

mozilla blog (95 kB)

13 µs

122 µs (9.4x)

whatwg spec (235 kB)

40.6 µs

365 µs (9.0x)

A bulk tag edit over each page’s <code>/<a>/<q> elements: remove() drops each match with its subtree, and strip_tags() unwraps each match but keeps its content. Both rewrites are destructive, so the timed call parses the page afresh – the string-to-result transform these helpers perform – and races each library’s own bulk tag helper. remove pairs with selectolax’s strip_tags (which drops matches with their content) and pyquery’s .remove(); strip_tags pairs with w3lib’s regex remove_tags (which keeps the content) and pyquery’s unwrap. turbohtml’s single C pass leads by roughly two to seven times, the margin widest on the smaller pages where the competitors’ per-match Python work has the least bulk text to hide behind.

remove (drop with content)

turbohtml

pyquery

selectolax

daring fireball (10 kB)

23.8 µs

155 µs (6.6x)

99.2 µs (4.2x)

ars technica (56 kB)

115 µs

323 µs (2.9x)

486 µs (4.3x)

mozilla blog (95 kB)

260 µs

1.1 ms (4.3x)

1.59 ms (6.2x)

whatwg spec (235 kB)

645 µs

3 ms (4.7x)

2.8 ms (4.4x)

strip_tags (unwrap keep content)

turbohtml

w3lib

pyquery

daring fireball (10 kB)

24 µs

70 µs (3.0x)

175 µs (7.3x)

ars technica (56 kB)

120 µs

315 µs (2.7x)

323 µs (2.7x)

mozilla blog (95 kB)

264 µs

618 µs (2.4x)

1.18 ms (4.5x)

whatwg spec (235 kB)

683 µs

1.64 ms (2.5x)

3.06 ms (4.5x)

Extraction

Pulling values out of a document, the idioms the parsel, pyquery, and w3lib migrations center on. First, reading every matched node’s @href and visible text off a pre-parsed page: parsel’s ::attr/::text getall and a pyquery .items() read against turbohtml selecting once and reading attr() and text off each node. turbohtml compiles the selector once and reads interned atoms, where parsel re-translates the CSS to XPath on libxml2 per call and pyquery boxes every match in a wrapper object, so it leads by twenty to nearly ninety times.

extract @href (per match)

turbohtml

parsel

pyquery

daring fireball (10 kB)

2.5 µs

65 µs (26.0x)

162 µs (64.8x)

ars technica (56 kB)

6 µs

142 µs (23.7x)

10 µs (1.7x)

mozilla blog (95 kB)

8.5 µs

212 µs (25.0x)

756 µs (89.0x)

whatwg spec (235 kB)

9 µs

275 µs (30.6x)

649 µs (72.1x)

extract text (per match)

turbohtml

parsel

pyquery

daring fireball (10 kB)

2.8 µs

66.6 µs (23.8x)

92 µs (32.9x)

ars technica (56 kB)

6.4 µs

129 µs (20.2x)

10 µs (1.6x)

mozilla blog (95 kB)

10 µs

189 µs (19.0x)

345 µs (34.5x)

whatwg spec (235 kB)

10.8 µs

247 µs (22.9x)

384 µs (35.6x)

Second, reading a document’s own URL hints: w3lib’s get_base_url and get_meta_refresh against turbohtml’s base_url() and meta_refresh(). Both parse the string each call; w3lib runs a regular-expression pass while turbohtml runs the WHATWG tree builder and reads the hint off the parsed <head>, so the tree builder still comes out ahead of the regex on this small document.

url hint

turbohtml

w3lib

base_url / get_base_url

1.3 µs

7.4 µs (5.7x)

meta_refresh / get_meta_refresh

1 µs

6.4 µs (6.4x)

Fluent chaining

A pyquery-style fluent chain over a pre-parsed tree: select every <a>, keep the linked ones, take the first, tag it, and read its href (turbohtml’s turbohtml.query.Query against pyquery, whose wrapper delegates to lxml). Both wrappers are thin Python over the underlying engine, so the gap is the engine’s: turbohtml’s selector and attribute primitives run in C, and the wrapper avoids a redundant de-duplication when the chain starts from one node, so it runs two to twenty-four times faster, depending on how much the page exercises the selector.

select, filter, tag, read

turbohtml

pyquery

daring fireball (10 kB)

3.7 µs

87.5 µs (23.7x)

ars technica (56 kB)

11.3 µs

22.8 µs (2.1x)

mozilla blog (95 kB)

17 µs

245 µs (14.5x)

whatwg spec (235 kB)

29 µs

313 µs (10.9x)

html.parser adapter

turbohtml.migration.stdlib.HTMLParser against the standard library’s html.parser.HTMLParser, both subclassed with the same minimal handler so the comparison is the parser and dispatch cost for the identical callback-driven programming model. The per-tag Python handler call is a floor both pay, so the margin is narrower than raw tokenization, but turbohtml’s C tokenizer feeding the dispatch still runs it three to four times faster.

feed and dispatch a page

turbohtml

html.parser

daring fireball (10 kB)

86.9 µs

308 µs (3.6x)

ars technica (56 kB)

374 µs

1.39 ms (3.8x)

mozilla blog (95 kB)

801 µs

3.2 ms (4.0x)

whatwg spec (235 kB)

2.5 ms

7.61 ms (3.1x)

CSS minification

turbohtml.clean.minify_css() against the CSS minifiers on PyPI, over the unminified source CSS these frameworks publish. Each minifier column pairs its output size with the time to produce it; the ratio in each cell is against turbohtml. Sizes are deterministic byte counts; times are the minimum of repeated runs. esbuild and tdewolff/minify are native minifiers invoked through their command line, so their millisecond timings are dominated by process startup; against them the clean comparison is output size, where turbohtml stays within a couple percent and comes out smaller on most of the corpus.

stylesheet

turbohtml

lightningcss

rcssmin

esbuild

tdewolff

csscompressor

cssmin

css-html-js-minify

size

time

size

time

size

time

size

time

size

time

size

time

size

time

size

time

normalize.css (6 kB)

1.8 kB

55 µs

1.8 kB (1.001x)

45.9 µs (0.9x)

1.8 kB (1.01x)

5.46 µs (0.1x)

1.8 kB (1.04x)

6.94 ms (127x)

1.8 kB (1.03x)

5.61 ms (103x)

1.8 kB (1.05x)

1.12 ms (20.5x)

1.8 kB (1.06x)

389 µs (7.1x)

1.8 kB (1.05x)

460 µs (8.4x)

pico.css (90 kB)

81.2 kB

1.43 ms

80.0 kB (0.99x)

1.51 ms (1.1x)

82.1 kB (1.01x)

205 µs (0.2x)

82.5 kB (1.02x)

9.13 ms (6.5x)

82.0 kB (1.01x)

6.54 ms (4.6x)

81.6 kB (1.01x)

36.7 ms (25.8x)

81.8 kB (1.01x)

219 ms (154x)

81.9 kB (1.01x)

221 ms (156x)

animate.css (93 kB)

73.8 kB

647 µs

68.8 kB (0.93x)

1.46 ms (2.3x)

75.7 kB (1.03x)

173 µs (0.3x)

73.5 kB (0.995x)

9.62 ms (14.9x)

74.4 kB (1.01x)

6.84 ms (10.6x)

75.7 kB (1.03x)

26.5 ms (41.0x)

75.8 kB (1.03x)

7.36 ms (11.4x)

75.8 kB (1.03x)

9.74 ms (15.1x)

foundation.css (164 kB)

132.9 kB

2.23 ms

—1

—1

136.7 kB (1.03x)

408 µs (0.2x)

131.8 kB (0.99x)

13 ms (5.9x)

136.0 kB (1.02x)

8.05 ms (3.7x)

136.4 kB (1.03x)

67 ms (30.0x)

136.3 kB (1.03x)

494 ms (222x)

136.5 kB (1.03x)

509 ms (228x)

bootstrap.css (274 kB)

229.6 kB

6.51 ms

228.7 kB (0.996x)

4.42 ms (0.7x)

233.2 kB (1.02x)

636 µs (0.1x)

233.0 kB (1.02x)

14.8 ms (2.3x)

231.8 kB (1.01x)

9.2 ms (1.5x)

234.2 kB (1.02x)

82 ms (12.6x)

232.4 kB (1.01x)

688 ms (106x)

234.3 kB (1.02x)

591 ms (90.8x)

bulma.css (745 kB)

681.4 kB

18.8 ms

674.3 kB (0.99x)

11.5 ms (0.7x)

680.0 kB (0.998x)

1.77 ms (0.1x)

689.2 kB (1.01x)

24.8 ms (1.4x)

685.3 kB (1.01x)

15.3 ms (0.9x)

681.3 kB (1.000x)

570 ms (30.4x)

679.3 kB (0.997x)

3.21 s (171x)

681.3 kB (1.000x)

2.85 s (152x)

1 Parsing stylesheet failed: Invalid media query at :577:29

csscompressor (the YUI port) and cssmin (its BSD descendant) rewrite values to their shortest form the way turbohtml does, but as pure-Python regex passes they turn quadratic on a large stylesheet and trail the C engine by 7x to 220x. rcssmin is a C extension and faster than turbohtml, though it only strips comments and whitespace, so it leaves a larger result everywhere except the custom-property-heavy bulma.css. css-html-js-minify is among the slowest of the set. The three pure-Python tools and rcssmin also break value safety: each rewrites the internal whitespace of a custom-property value, which CSS Variables 1 §2 keeps as the literal token stream that var() splices verbatim and getPropertyValue() reads back byte-exact, and cssmin and css-html-js-minify collapse whitespace inside strings, so their output can change the cascade where turbohtml’s round-trips. That rewrite is also the only reason rcssmin and cssmin end 0.2% to 0.3% ahead on bulma.css, whose declarations are almost entirely custom properties.

lightningcss, the Rust binding, is a cascade-aware optimizer: it drops declarations overridden elsewhere in the sheet and rewrites syntax for a browser-target set, so it reaches a smaller size than turbohtml on most of the corpus (turbohtml comes out ahead on normalize.css). That target-dependent optimization is the same idea as turbohtml’s baseline option carried further, and it is in scope. Its Rust engine keeps pace on time – faster than turbohtml on the largest sheets, about twice slower on animate.css where the cascade pass does the most work – but it rejects foundation.css with a parse error on a media query the WHATWG recovery rules accept, where turbohtml minifies all six. turbohtml gives the smallest value-safe output at the most compatible baseline and recovers from malformed input.

JavaScript minification

turbohtml.clean.minify_js() against the PyPI JavaScript minifiers it replaces – rjsmin (a regex substitution), jsmin (Crockford’s character state machine), and calmjs.parse (a full ES5 parser with an obfuscating printer) – and the industry’s native minifiers as the size bar: terser (the JavaScript ecosystem’s reference), esbuild, and tdewolff/minify, each invoked through its command line. The inputs are real un-minified libraries, a size ladder every tool parses. turbohtml renames every local binding (function and class declarations included) and runs the structural folds, so it beats calmjs.parse’s heavier global obfuscation on size everywhere while running fifty to a hundred times faster, and its output lands within one percent of terser, esbuild, and tdewolff – the best minifiers available. It runs in-process, where each native tool pays its runtime’s process startup per file, so it finishes ahead end to end; only rjsmin is faster, and it does so by leaving output close to twice the size. Each cell pairs a minifier’s output size with the time to produce it; both ratios are against turbohtml.

input

turbohtml

tdewolff

esbuild

rjsmin

terser

calmjs.parse

css-html-js-minify

jsmin

size

time

size

time

size

time

size

time

size

time

size

time

size

time

size

time

underscore 1.13 (67 kB)

19.3 kB

2.24 ms

19.3 kB (1.000x)

5.27 ms (2.4x)

19.3 kB (1.002x)

7.52 ms (3.4x)

34.0 kB (1.76x)

83.1 µs (0.1x)

19.2 kB (0.996x)

150 ms (67.0x)

20.9 kB (1.08x)

107 ms (47.6x)

34.0 kB (1.76x)

4.67 ms (2.1x)

34.0 kB (1.76x)

6.2 ms (2.8x)

backbone 1.6 (79 kB)

24.8 kB

1.17 ms

24.6 kB (0.99x)

5.57 ms (4.8x)

24.7 kB (0.998x)

7.69 ms (6.6x)

35.2 kB (1.42x)

71.2 µs (0.1x)

24.6 kB (0.99x)

162 ms (139x)

25.9 kB (1.05x)

125 ms (107x)

35.2 kB (1.42x)

4.91 ms (4.2x)

35.2 kB (1.42x)

6.66 ms (5.7x)

jquery 3.7 (279 kB)

87.8 kB

10.8 ms

87.4 kB (0.996x)

10 ms (1.0x)

87.7 kB (0.999x)

14 ms (1.4x)

141.1 kB (1.61x)

380 µs (0.1x)

87.0 kB (0.99x)

271 ms (25.2x)

93.2 kB (1.06x)

513 ms (47.7x)

148.0 kB (1.69x)

20.2 ms (1.9x)

141.4 kB (1.61x)

25.7 ms (2.4x)

lodash 4.17 (531 kB)

71.9 kB

10.1 ms

71.9 kB (1.000x)

11.9 ms (1.2x)

72.8 kB (1.01x)

16.7 ms (1.7x)

148.8 kB (2.07x)

593 µs (0.1x)

71.1 kB (0.99x)

295 ms (29.2x)

77.0 kB (1.07x)

506 ms (50.1x)

140.3 kB (1.95x)

33.7 ms (3.4x)

148.8 kB (2.07x)

35 ms (3.5x)

Encoding detection

turbohtml.detect.detect() against the encoding detectors it replaces: chardet (the pure-Python prober ensemble), charset-normalizer (decode-and-score, what requests uses), and faust-cchardet (the maintained C binding of uchardet; the original cchardet stops compiling at Python 3.11). turbohtml resolves certain input – a byte-order mark, a <meta> declaration, valid UTF-8, pure ASCII – structurally before any scoring, which is where the 40x-1800x rows come from, and its chardetng frequency scoring keeps declaration-less single-byte text about 3x ahead of chardet. The one workload it loses is CJK-heavy bytes, where each CJK candidate drives a CPython incremental codec and uchardet’s native tables stay ahead.

detect a byte stream’s encoding

turbohtml

faust-cchardet

charset-normalizer

chardet

ascii (1 kB)

1.96 µs

926 ns (0.5x)

46.6 µs (23.8x)

121 µs (61.4x)

utf-8 russian (4 kB)

3.77 µs

4.33 µs (1.2x)

348 µs (92.3x)

153 µs (40.6x)

windows-1251 russian (4 kB)

206 µs

323 µs (1.6x)

312 µs (1.6x)

671 µs (3.3x)

windows-1252 french (4 kB)

206 µs

368 µs (1.8x)

986 µs (4.8x)

701 µs (3.5x)

shift_jis japanese (4 kB)

773 µs

34.2 µs (0.1x)

608 µs (0.8x)

597 µs (0.8x)

utf-8 page (95 kB)

705 ns

54.5 µs (77.3x)

903 µs (1281x)

1.28 ms (1817x)