Performance

Every number here comes from pyperf on CPython 3.14.6 (a release build) on an Apple M4 running macOS 26. pyperf runs each case in isolated worker processes and reports the mean; the harness prints the run-to-run standard deviation beside it as ±N% so a real gap reads apart from noise, and these tables quote the mean. For the lowest-noise figures, tune the machine with pyperf system tune first (and sudo pyperf system reset after); pyperf options like --rigorous or --affinity pass straight through after the tox -e bench command. Operations that mutate the tree they are handed – the edits, the content setters, link absolutization – are timed on a fresh parse rebuilt before each iteration (the rebuild itself untimed), so the figure is the repeatable cost of the mutation alone rather than a tree a prior iteration already changed; read-path operations reuse one cached parse and time only the query. The corpora are real documents: Project Gutenberg’s War and Peace, the WHATWG HTML specification source, the ECMAScript specification, a size-weighted sample of web-platform-tests pages for the parse and tokenize suites, and, for the read-path suites, real saved web pages – a blog, a news article, and a product blog from the mozilla/readability test corpus – so the selector, link, and edit operations run against genuine nested structure rather than the layout fixtures, which carry none. The harness benchmarks each competitor in its own isolated uv venv – turbohtml in a venv of its own as the shared baseline – so one library’s dependency pins never perturb another’s. Every table below is one harness operation, so each is reproducible with tox -e bench <command>, where the command is core (turbohtml’s own baseline for every operation), an operation name (the cross-competitor table), a package name (that competitor’s own report), or all. The numbers in these tables come from the --pgo baseline: turbohtml built with the shipped profile-guided, link-time-optimized release recipe (tox -e bench -- --pgo all), so each figure reads as what a release ships rather than a plain build. The default baseline is a plain wheel, which builds quickly for iterating; the --pgo build costs much more time. Most operations are a single call; a few aggregate workloads (build, build-e) sweep a size, and the construct and emit breakdowns decompose that write path into the constructor and the serializer in isolation. Numbers vary with input and hardware.

To refresh these tables, run the sweep into a scratch directory and let the generators rewrite the committed feeds; the harness names its output for the operation, which is not what this guide calls its tables, so never copy the files across by hand:

tox -e bench -- --pgo --table-json /tmp/feeds all
python -m bench.docs_feeds /tmp/feeds docs/development/bench
python -m bench.migration /tmp/feeds docs/migration/bench docs tools/bench/competitors

Escaping

turbohtml.escape() against the standard library’s html.escape() (the stdlib column), dominate’s text escape, and the nh3 ammonia binding. It gains the most on text that needs little escaping, where the SIMD scan classifies sixteen bytes at a time and copies clean stretches in bulk: on 4 MiB of no-op prose it runs 22 times faster than html.escape and 65 times faster than nh3. The gap narrows to two to four times on tiny strings and escape-dense markup, where call overhead and the escaping itself dominate.

escape

turbohtml

dominate

stdlib

nh3

tiny plain (64 B)

58.6 ns

111 ns (2.0x ±2%)

126 ns (2.2x ±3%)

233 ns (4.0x ±3%)

medium markup (4 KiB)

2.43 µs

6.48 µs (2.7x ±2%)

7.94 µs (3.3x ±2%)

8.09 µs (3.4x ±2%)

no-op prose (4 MiB)

123 µs

2.19 ms (17.8x ±1%)

2.74 ms (22.3x ±2%)

8 ms (64.9x ±1%)

book text (3 MiB)

654 µs

2.14 ms (3.3x ±1%)

2.89 ms (4.5x ±1%)

9.3 ms (14.3x ±1%)

book HTML (4 MiB)

1.31 ms

4.33 ms (3.4x ±3%)

5.08 ms (3.9x ±1%)

14.7 ms (11.3x ±3%)

spec HTML, dense (4 MiB)

5.34 ms

12.9 ms (2.5x ±5%)

13.9 ms (2.7x ±1%)

14 ms (2.7x ±1%)

UCS-2 plain (4 MiB)

844 µs

2.11 ms (2.6x ±2%)

2.65 ms (3.2x ±2%)

14 ms (16.6x ±2%)

UCS-2 markup (4 MiB)

5.79 ms

9.87 ms (1.8x ±1%)

11.8 ms (2.1x ±1%)

14.7 ms (2.6x ±2%)

UCS-4 plain (4 MiB)

976 µs

4.54 ms (4.7x ±1%)

5.69 ms (5.9x ±1%)

17.2 ms (17.7x ±4%)

UCS-4 markup (4 MiB)

7.06 ms

16.6 ms (2.4x ±3%)

20.8 ms (3.0x ±3%)

18.1 ms (2.6x ±4%)

Markup (escaping)

turbohtml.migration.markupsafe.escape() against markupsafe’s own C escape, both returning a Markup. The inputs are the small, mostly-clean strings a template engine interpolates under autoescape, markupsafe’s hottest path. turbohtml builds the safe string in C in a single call, where markupsafe pays a Python escape frame and Markup construction per call, so it runs two and a half to nearly four times faster across the clean and escape-heavy inputs.

markupsafe-compatible escape

turbohtml

markupsafe

clean (8 B)

80.5 ns

251 ns (3.2x ±5%)

clean (32 B)

93.4 ns

285 ns (3.1x ±9%)

clean (256 B)

164 ns

604 ns (3.7x ±10%)

name with ‘ and &

111 ns

284 ns (2.6x ±4%)

escape-heavy markup

181 ns

449 ns (2.5x ±4%)

The other Markup operations race markupsafe’s own Markup of the same method. striptags and unescape run on turbohtml’s tokenizer and HTML5 reference resolution where markupsafe scans with a regex, and format and join escape each untrusted operand through the same C escape.

Markup operations

turbohtml

markupsafe

striptags

1.57 µs

3.05 µs (2.0x ±11%)

unescape

245 ns

1.3 µs (5.3x ±7%)

format (escapes operands)

2.25 µs

2.74 µs (1.3x ±4%)

join (escapes operands)

821 ns

1.6 µs (2.0x ±7%)

Linkify

turbohtml.clean.linkify() against bleach’s linkify, the HTML-aware linkifier it succeeds, and lxml-html-clean’s autolink. All three parse the HTML and rewrite it. turbohtml’s C candidate scan and its own tree carry it past bleach’s html5lib pass by six to twenty times. It leads lxml’s autolink on the comment and 4 KiB markup inputs and trails it on the plain 1 KiB prose row (0.6x), though that row is not a like-for-like comparison: autolink only rewrites URLs already inside markup and never linkifies an email address, so on plain prose it produces no links at all where turbohtml produces thirty. Its figure there is the cost of finding nothing.

linkify HTML

turbohtml

lxml-html-clean1

bleach

comment (1 link, 1 email)

2.83 µs

6.93 µs (2.5x ±15%)

56.9 µs (20.1x ±5%)

prose (1 KiB)

42.1 µs

23 µs (0.6x ±2%)

278 µs (6.6x ±4%)

markup (4 KiB)

111 µs

140 µs (1.3x ±2%)

1.62 ms (14.6x ±2%)

1 lxml-html-clean links URLs inside existing markup and never linkifies email addresses; on the prose case it produces no links at all, so that timing is the cost of finding nothing

A single C walk creates wrappers only for eligible text and anchor targets; Python no longer visits every node. CodSpeed tracks a text-heavy tree, 2,000 small text nodes, and 2,000 empty elements. Separate cases cover skip-tag pruning and callback-heavy HTML. Callbacks run in document order after target collection releases the tree lock.

The detection primitive on its own, turbohtml.clean.LinkDetector.find() against LinkifyIt().match and has_link() against LinkifyIt().test, scans a run of plain text without rewriting HTML. Both libraries stop on the first valid match; turbohtml allocates no span list. The large-tail case starts with a link followed by 220 KiB of prose to catch a return to full-input scanning.

detect links in text

turbohtml

linkify-it-py

find comment (1 link, 1 email)

719 ns

45 µs (62.7x ±7%)

find prose (1 KiB)

9.34 µs

477 µs (51.1x ±5%)

has_link comment

320 ns

32.8 µs (103x ±11%)

has_link prose (1 KiB)

3.33 µs

9.18 µs (2.8x ±13%)

Sanitize

turbohtml.clean.sanitize() against four sanitizers. Three share its allowlist model, where only listed tags and attributes survive, so a vector nobody anticipated is dropped by default: nh3 (the Rust ammonia binding), bleach (its end-of-life predecessor, on html5lib), and html-sanitizer (an allowlist over lxml). The fourth, lxml-html-clean (the externalized lxml.html.clean.Cleaner), is a blocklist: it strips the constructs it knows are dangerous and lets the rest through, a model lxml itself flagged as hard to keep safe. The inputs are realistic user content with a few disallowed tags and a dangerous attribute mixed in. turbohtml runs the whole filtering walk in C and leads every alternative, but the model matters more than the microseconds. Prefer an allowlist, since a blocklist passes anything it did not think to name.

sanitize

turbohtml

nh3

lxml-html-clean

html-sanitizer

bleach

comment

2.22 µs

8.6 µs (3.9x ±17%)

31.9 µs (14.4x ±18%)

74.6 µs (33.7x ±14%)

138 µs (62.3x ±23%)

post 4 KiB

57.9 µs

202 µs (3.5x ±10%)

790 µs (13.7x ±11%)

2.5 ms (43.3x ±14%)

3.33 ms (57.6x ±18%)

Template-safe sanitizing (Policy.strip_template_markers, collapsing {{ }}/${ }/<% %> so the output cannot re-inject through a template engine) has no allowlist-sanitizer analog in Python; the reference is DOMPurify’s SAFE_FOR_TEMPLATES, which runs in JavaScript. Reaching it from Python means shelling out to Node, where each call spins up a DOM before it sanitizes, so the figure below is that end-to-end per-document cost, not a pure-algorithm comparison. turbohtml folds the same transform into its C walk and pays neither the process nor the DOM.

sanitize (template-safe)

turbohtml

DOMPurify

templated 4 KiB

48 µs

428 ms (8929x ±14%)

Markdown

turbohtml.Node.to_markdown() against markdownify (on BeautifulSoup) and html2text (a streaming HTMLParser subclass). All three take an HTML string and return Markdown, so each parses first; turbohtml parses to the WHATWG tree and walks it in C, where the others build and convert in Python. The single C pass converts a page in a few microseconds, two orders of magnitude ahead of both. The configured row turns the option surface on in all three (underscore emphasis, reference links, padded tables, full escaping), where turbohtml stays 48 times ahead of html2text and 125 times ahead of markdownify.

HTML to Markdown

turbohtml

html2text

markdownify

article (2 KiB)

14.8 µs

840 µs (56.7x ±17%)

2.06 ms (139x ±15%)

list (4 KiB)

25 µs

1.66 ms (66.2x ±12%)

4.49 ms (180x ±17%)

table (4 KiB)

29.4 µs

1.54 ms (52.6x ±6%)

4.86 ms (166x ±12%)

configured (4 KiB)

38.7 µs

1.86 ms (48.2x ±15%)

4.85 ms (126x ±14%)

google_doc (4 KiB)

28.8 µs

914 µs (31.8x ±18%)

—1

1 no equivalent operation

The google_doc row reads the inline-CSS styling a Google Docs export carries (html2text’s google_doc mode) and runs 32 times faster; markdownify has no equivalent.

Structured data

turbohtml.Document.structured_data() against extruct, the scraper toolkit it succeeds, extracting JSON-LD, Microdata, and OpenGraph from a product page that carries all three. Both start from the raw HTML string, so each parses first; extruct builds an lxml tree and runs a separate extractor per syntax, where turbohtml parses to the WHATWG tree and gathers every format in one C walk, handing back the typed StructuredData record. The single pass runs roughly nine to eleven times faster.

structured-data extraction

turbohtml

extruct

product

8.08 µs

107 µs (13.3x ±13%)

catalog 8 KiB

74.8 µs

863 µs (11.6x ±13%)

Tables

turbohtml.Node.tables() and turbohtml.Element.records() against pandas’s read_html, the one-call table reader scrapers reach for. Both parse the HTML and extract every <table>, resolving rowspan and colspan into a rectangular grid; read_html returns a DataFrame per table and pulls in NumPy, where turbohtml runs the cell-grid walk in C and hands back plain list and dict objects with no added dependency. The rows row times tables() (every table as list[list[str]]) and the records row times records() (the first table keyed by its header), each over a four-column table of 10, 100, and 1,000 rows. The single C pass leads from roughly thirty times on the thousand-row table to over a hundred and sixty on the ten-row table, where pandas pays its fixed per-frame construction cost.

extract table grids

turbohtml

pandas

rows (10 rows)

9.43 µs

575 µs (61.1x ±19%)

records (10 rows)

9.62 µs

751 µs (78.1x ±17%)

rows (100 rows)

67.8 µs

1.78 ms (26.4x ±22%)

records (100 rows)

77 µs

2.19 ms (28.5x ±18%)

rows (1000 rows)

733 µs

12.3 ms (16.9x ±12%)

records (1000 rows)

769 µs

13.3 ms (17.4x ±15%)

Article extraction

turbohtml.Node.article() against trafilatura, readability-lxml, newspaper3k, goose3, readabilipy, and news-please, the article extractors it succeeds. Each scores the dominant content body and (trafilatura, newspaper3k, goose3, and news-please) harvests the page metadata beside it; the lxml-backed four build their tree in Python first, readabilipy’s Python mode parses with html5lib into BeautifulSoup and cleans without scoring, news-please merges the votes of several such extractors, and turbohtml does the scoring and the harvest in one C pass over the parsed tree. The inputs are full pages – navigation, a scored article, and a footer – so the boilerplate the heuristic discounts is part of the measured cost.

article extraction

turbohtml

readability-lxml

trafilatura

newspaper3k

goose3

news-please

readabilipy

post (4 KiB)

14.7 µs

843 µs (57.2x ±9%)

940 µs (63.8x ±12%)

3.14 ms (213x ±12%)

3.44 ms (234x ±11%)

9.89 ms (672x ±18%)

8.37 ms (568x ±15%)

longform (16 KiB)

46.7 µs

2.29 ms (49.1x ±13%)

2.62 ms (56.2x ±9%)

8.83 ms (189x ±11%)

12.2 ms (261x ±11%)

24.8 ms (531x ±14%)

30.5 ms (652x ±27%)

Boilerplate classification

turbohtml.extract.boilerplate() against justext and boilerpy3, the per-block boilerplate classifiers. All three segment the page into units and mark each good or boilerplate; justext scores every paragraph in Python over an lxml tree (length, link density, stopword density), boilerpy3 classifies the blocks of its own SAX stream with boilerpipe’s rules, and turbohtml scores the tree once in C and classifies the units in a thin Python layer. The inputs are the article-extraction pages, so the navigation and footer each classifier must reject are part of the measured cost.

paragraph boilerplate classification

turbohtml

boilerpy3

justext

post (4 KiB)

35.1 µs

373 µs (10.7x ±4%)

1.17 ms (33.4x ±5%)

longform (16 KiB)

134 µs

1.24 ms (9.3x ±5%)

4.76 ms (35.6x ±5%)

Date extraction

turbohtml.extract.dates() against htmldate, the standalone publication-date finder, and the article extractors trafilatura, newspaper3k, goose3, and news-please that surface a date beside the body text. All read the same signals – publication/modification <meta> tags, JSON-LD, <time> elements, and a date in the URL – and are parse-bound; htmldate builds an lxml tree, turbohtml the WHATWG tree. turbohtml’s early-exit over the structured signals runs 2.9 to 3.3 times faster than htmldate on the real pages and 20 to 24 times faster than trafilatura. The synthetic 100 meta candidates row – a page stacked with a hundred date-like <meta> tags – was the one case turbohtml lost, since it weighed every candidate where htmldate and trafilatura stopped early; moving that weighing into C turns it around, and turbohtml now leads the row too, 3.0 times over htmldate and 4.1 times over trafilatura.

publication-date extraction

turbohtml

htmldate1

trafilatura2

newspaper3k

goose3

news-please

post (4 KiB)

5.93 µs

16.9 µs (2.9x ±1%)

118 µs (19.9x ±2%)

2.12 ms (358x ±3%)

2.5 ms (422x ±16%)

5.93 ms (1000x ±4%)

longform (16 KiB)

10.3 µs

34 µs (3.4x ±2%)

250 µs (24.3x ±2%)

6.14 ms (597x ±2%)

8.27 ms (804x ±1%)

15.9 ms (1545x ±2%)

100 meta candidates

18.3 µs

54.6 µs (3.0x ±2%)

73.2 µs (4.1x ±6%)

2.59 ms (142x ±2%)

4.19 ms (230x ±2%)

8.77 ms (480x ±4%)

1 htmldate returns no date for the 100-candidate case, so its timing there is the cost of giving up rather than of finding the date turbohtml reports

2 trafilatura returns no date for the 100-candidate case, so its timing there is the cost of giving up rather than of finding the date turbohtml reports

Unescaping

turbohtml.unescape() against html.unescape() (the stdlib column), w3lib’s replace_entities, the Scrapy helper that resolves the same references, and dominate’s util.unescape. It gains the most on entity-heavy input, where the standard library pays a Python call per match and w3lib runs a regular-expression substitution with a Python callback per match; turbohtml hops between & occurrences in C and bulk-copies the clean spans between references, so it leads html.unescape by up to 16 times and w3lib by up to 22 times on the reference-dense inputs. dominate sits in that same range on the small strings, but its scan turns multi-megabyte input into whole seconds, trailing by 525 times on the 4 MiB book and past 2,600 on the escaped copy.

unescape

turbohtml

stdlib

w3lib

dominate

tiny plain (64 B)

35.9 ns

39.8 ns (1.2x ±2%)

258 ns (7.2x ±5%)

206 ns (5.8x ±2%)

medium dense refs (4 KiB)

4.92 µs

74.1 µs (15.1x ±1%)

109 µs (22.2x ±1%)

105 µs (21.3x ±1%)

numeric refs (4 KiB)

5.17 µs

82.8 µs (16.1x ±1%)

90.8 µs (17.6x ±1%)

98.8 µs (19.2x ±2%)

book HTML, real refs (4 MiB)

1.95 ms

9.16 ms (4.7x ±9%)

13.1 ms (6.8x ±3%)

1.02 s (525x ±9%)

escaped book HTML (5 MiB)

1.7 ms

19.7 ms (11.6x ±6%)

35.2 ms (20.7x ±2%)

4.45 s (2615x ±4%)

dense refs (4 MiB)

5.6 ms

72.6 ms (13.0x ±4%)

118 ms (21.1x ±4%)

14.7 s (2633x ±10%)

UCS-2 refs (4 MiB)

1.89 ms

18.3 ms (9.7x ±2%)

27.1 ms (14.4x ±2%)

5.9 s (3122x ±7%)

Tokenizing

turbohtml.tokenize() against html.parser.HTMLParser (the stdlib column, driven with no-op handlers) and html5lib’s pure-Python tokenizer. The closest case is a document dominated by a single text node (the text-heavy prose row, 4.8x), where the standard library’s regex performs one C scan; wherever markup appears, the state machine runs roughly eight to sixteen times faster than html.parser and 22 to 240 times faster than html5lib.

tokenize

turbohtml

stdlib

html5lib

typical markup

28.6 µs

426 µs (15.0x ±2%)

806 µs (28.3x ±2%)

text-heavy prose

595 ns

2.79 µs (4.7x ±3%)

143 µs (240x ±1%)

attribute-heavy

18.3 µs

298 µs (16.3x ±3%)

812 µs (44.4x ±1%)

script-heavy

11.6 µs

147 µs (12.8x ±2%)

486 µs (42.0x ±1%)

entity-heavy

17.3 µs

195 µs (11.3x ±3%)

1.21 ms (70.3x ±3%)

wpt tiny (0.6 kB)

1.45 µs

17.5 µs (12.2x ±2%)

47.8 µs (33.0x ±2%)

wpt small (4 kB)

11.9 µs

168 µs (14.2x ±2%)

423 µs (35.7x ±1%)

wpt medium (9.6 kB)

27.7 µs

367 µs (13.3x ±2%)

1.15 ms (41.5x ±1%)

wpt large (92 kB)

329 µs

3.93 ms (12.0x ±3%)

8.97 ms (27.3x ±1%)

wpt CJK (124 kB)

563 µs

8.34 ms (14.9x ±3%)

22.1 ms (39.3x ±1%)

whatwg spec (235 kB)

651 µs

7.47 ms (11.5x ±3%)

19.1 ms (29.4x ±1%)

ecmascript spec (3 MB)

6.27 ms

53.2 ms (8.5x ±2%)

179 ms (28.6x ±1%)

whatwg spec source (7.9 MB)

37 ms

369 ms (10.0x ±1%)

849 ms (23.0x ±1%)

Parsing

turbohtml.parse() builds a full WHATWG document tree, against the other Python tree builders: lxml and parsel and pyquery (all over libxml2), selectolax and resiliparse (both wrapping lexbor), html5-parser (the C gumbo binding), BeautifulSoup over each of its tree builders, and html5lib. turbohtml leads resiliparse by 1.2 to 3.4 times, runs 2.7 to 6.2 times faster than lxml, parsel, pyquery, and selectolax, 4.9 to 12.6 times faster than html5-parser, and 31 to 99 times faster than html5lib and BeautifulSoup, while building the WHATWG tree that lxml’s libxml2 does not.

resiliparse stays closest because its HTMLTree.parse is a thin call straight into lexbor’s native tree, while selectolax wraps that same engine behind a heavier object layer; the comparison here is parsing only. resiliparse’s wider toolkit, boilerplate and main-content extraction, language detection, and the encoding and archive utilities it ships for large-scale web-crawl processing, sits outside turbohtml’s scope. html5-parser wraps gumbo, the C WHATWG parser Google released; it is read-oriented and archived upstream, and it trails by five to thirteen times above. turbohtml is the maintained, mutable, typed alternative to that lineage.

parse to a tree

turbohtml

resiliparse

lxml

pyquery

parsel

selectolax

html5-parser

BeautifulSoup (lxml)

BeautifulSoup (html.parser)

html5lib

wpt tiny (0.6 kB)

1.53 µs

5.21 µs (3.4x ±7%)

4.71 µs (3.1x ±5%)

6.19 µs (4.1x ±3%)

9.57 µs (6.3x ±4%)

9.51 µs (6.3x ±6%)

16.8 µs (11.0x ±3%)

103 µs (67.4x ±6%)

103 µs (67.3x ±7%)

152 µs (99.4x ±5%)

wpt small (4 kB)

12.2 µs

17.2 µs (1.5x ±4%)

36.1 µs (3.0x ±3%)

38.4 µs (3.2x ±4%)

43.9 µs (3.6x ±4%)

56 µs (4.6x ±3%)

93.4 µs (7.7x ±3%)

525 µs (43.0x ±5%)

690 µs (56.5x ±4%)

924 µs (75.6x ±4%)

wpt medium (9.6 kB)

32.5 µs

37.9 µs (1.2x ±6%)

98 µs (3.1x ±7%)

101 µs (3.2x ±7%)

108 µs (3.4x ±8%)

137 µs (4.3x ±6%)

219 µs (6.8x ±7%)

999 µs (30.8x ±9%)

1.31 ms (40.2x ±7%)

2.12 ms (65.3x ±6%)

wpt large (92 kB)

276 µs

384 µs (1.4x ±2%)

862 µs (3.2x ±3%)

884 µs (3.2x ±3%)

867 µs (3.2x ±2%)

1.26 ms (4.6x ±2%)

2.88 ms (10.5x ±3%)

16.4 ms (59.3x ±8%)

23 ms (83.3x ±8%)

25.4 ms (91.8x ±2%)

wpt CJK (124 kB)

532 µs

712 µs (1.4x ±6%)

1.89 ms (3.6x ±10%)

2.02 ms (3.9x ±7%)

1.94 ms (3.7x ±7%)

2.96 ms (5.6x ±5%)

3.41 ms (6.5x ±6%)

23.1 ms (43.5x ±9%)

33.3 ms (62.7x ±12%)

42.6 ms (80.2x ±10%)

whatwg spec (235 kB)

519 µs

687 µs (1.4x ±3%)

1.68 ms (3.3x ±4%)

1.68 ms (3.3x ±3%)

1.74 ms (3.4x ±3%)

2.45 ms (4.8x ±3%)

6.55 ms (12.7x ±4%)

29.1 ms (56.1x ±7%)

37.8 ms (72.9x ±12%)

46.4 ms (89.6x ±8%)

common tags (13 kB)

78.5 µs

99.1 µs (1.3x ±4%)

213 µs (2.8x ±4%)

225 µs (2.9x ±3%)

220 µs (2.9x ±8%)

333 µs (4.3x ±3%)

387 µs (5.0x ±3%)

4.88 ms (62.2x ±4%)

6.99 ms (89.1x ±7%)

6.62 ms (84.4x ±5%)

Fragment parsing

turbohtml.parse_fragment() parses an innerHTML-style snippet in a container’s context rather than a whole document, against lxml’s lxml.html.fromstring and html5lib’s parseFragment. The input is a table-row fragment parsed in its <tbody> context, where the WHATWG algorithm’s table rules apply. turbohtml runs the same C engine it uses for whole documents, so it parses the fragment nearly four times faster than lxml and roughly eighty-eight times faster than the pure-Python html5lib.

parse a fragment

turbohtml

lxml

html5lib

table-row fragment (2 kB)

10.2 µs

37.5 µs (3.7x ±2%)

886 µs (86.7x ±3%)

Querying

Each library parses the document once, then the timed call runs one query. find collects every <a> element the way each library reaches for it (turbohtml’s find_all(), resiliparse’s and selectolax’s lexbor selectors, lxml’s XPath findall, parsel’s and pyquery’s and BeautifulSoup’s selectors, and soupsieve directly). A tag-only query resolves the name to an interned atom and walks the subtree comparing integers, with no per-element string built and no matcher dispatch. It stays ahead of resiliparse’s lexbor pass by 1.3 times on the small blog widening to 22 times on the spec, leads lxml’s C XPath engine by 14 to 24 times, and runs 12 to over 1,400 times ahead of pyquery, selectolax, parsel, BeautifulSoup, and soupsieve.

A first-result query walks until its match when the document has no tag index. An uncapped find_all builds the whole-document index for later queries; find and find_all(..., limit=1) reuse it after that point. The cold-tree benchmark records hit positions and misses. It measures uncapped and limited collection, plus peak resident memory.

find every anchor

turbohtml

resiliparse

lxml

selectolax

pyquery

parsel

BeautifulSoup (lxml)

BeautifulSoup (html.parser)

soupsieve

daring fireball (10 kB)

372 ns

485 ns (1.4x ±3%)

5.1 µs (13.8x ±1%)

5.67 µs (15.3x ±2%)

9.77 µs (26.3x ±1%)

20.6 µs (55.5x ±1%)

13.2 µs (35.7x ±1%)

13.6 µs (36.5x ±2%)

72.5 µs (195x ±1%)

ars technica (56 kB)

811 ns

2.9 µs (3.6x ±4%)

13.3 µs (16.4x ±3%)

15.9 µs (19.7x ±2%)

20 µs (24.8x ±2%)

46.5 µs (57.4x ±2%)

48.9 µs (60.4x ±4%)

47.6 µs (58.8x ±2%)

270 µs (333x ±2%)

mozilla blog (95 kB)

1.12 µs

10 µs (9.0x ±5%)

20.1 µs (18.0x ±2%)

28.5 µs (25.5x ±3%)

29.5 µs (26.4x ±2%)

67.7 µs (60.5x ±2%)

99.6 µs (89.0x ±2%)

104 µs (92.6x ±6%)

574 µs (513x ±2%)

whatwg spec (235 kB)

1.3 µs

30.2 µs (23.3x ±3%)

31.4 µs (24.3x ±3%)

75.3 µs (58.0x ±2%)

60 µs (46.3x ±2%)

105 µs (80.6x ±2%)

342 µs (264x ±1%)

349 µs (269x ±5%)

1.94 ms (1491x ±2%)

select runs the CSS selector div a[href] (turbohtml’s select(), resiliparse’s and selectolax’s css, lxml’s cssselect, parsel’s css, pyquery, and BeautifulSoup’s soupsieve). Because turbohtml compiles the selector against the tree once and then matches by comparing interned integer atoms, it stays in the low microseconds across these pages. resiliparse’s lexbor engine stays closest at 3.4 to 19 times, selectolax next at 13 to 45 times. lxml and parsel re-translate the selector to XPath through cssselect on every call, which scales with the document and trails by roughly fifty times on the small blog up to nearly eight hundred times on the spec, with pyquery tracking them; soupsieve and BeautifulSoup are hundreds to more than fifteen hundred times behind.

select div a[href]

turbohtml

resiliparse

selectolax

lxml

pyquery

parsel

soupsieve

BeautifulSoup (html.parser)

BeautifulSoup (lxml)

daring fireball (10 kB)

595 ns

2.04 µs (3.5x ±2%)

7.58 µs (12.8x ±1%)

29.3 µs (49.3x ±1%)

29.8 µs (50.1x ±2%)

32.8 µs (55.2x ±2%)

139 µs (234x ±1%)

171 µs (288x ±2%)

170 µs (287x ±1%)

ars technica (56 kB)

1.43 µs

6.14 µs (4.3x ±2%)

20.2 µs (14.1x ±2%)

126 µs (88.2x ±1%)

127 µs (88.5x ±1%)

145 µs (102x ±1%)

463 µs (324x ±1%)

566 µs (396x ±2%)

567 µs (396x ±1%)

mozilla blog (95 kB)

2.01 µs

12.4 µs (6.2x ±3%)

34.1 µs (17.0x ±1%)

803 µs (400x ±1%)

801 µs (400x ±1%)

832 µs (415x ±1%)

869 µs (433x ±1%)

1.07 ms (531x ±1%)

1.07 ms (534x ±1%)

whatwg spec (235 kB)

1.76 µs

34.1 µs (19.4x ±2%)

79.4 µs (45.1x ±1%)

1.37 ms (775x ±2%)

1.42 ms (804x ±6%)

1.41 ms (797x ±2%)

2.27 ms (1286x ±1%)

2.78 ms (1576x ±2%)

2.78 ms (1577x ±1%)

The relational :has() pseudo-class is the costliest selector to evaluate, since a naive matcher rescans each candidate’s subtree. turbohtml runs div:has(a) against the same pages and leads every alternative: resiliparse and selectolax by five to twenty times, lxml and parsel by tens of times on the smaller pages, narrowing to single digits on the link-dense mozilla blog where the relational match itself does real work, while soupsieve and BeautifulSoup trail by hundreds of times throughout. The matcher walks each anchor’s descendants once and skips the sibling scan for descendant and child relationships, so the relational lookup keeps the same interned-atom comparison the flat selectors use.

select div:has(a)

turbohtml

resiliparse

selectolax

lxml

parsel

soupsieve

BeautifulSoup (html.parser)

BeautifulSoup (lxml)

daring fireball (10 kB)

262 ns

2.44 µs (9.4x ±3%)

4.95 µs (19.0x ±2%)

14.6 µs (55.8x ±3%)

11.2 µs (42.9x ±4%)

89.7 µs (343x ±3%)

112 µs (428x ±3%)

112 µs (430x ±3%)

ars technica (56 kB)

1.26 µs

10.9 µs (8.7x ±2%)

17.8 µs (14.1x ±2%)

27.7 µs (22.0x ±2%)

40.7 µs (32.3x ±2%)

400 µs (318x ±2%)

489 µs (388x ±2%)

492 µs (391x ±2%)

mozilla blog (95 kB)

8.54 µs

39 µs (4.6x ±3%)

51.9 µs (6.1x ±2%)

60.5 µs (7.1x ±3%)

84 µs (9.9x ±2%)

1.77 ms (207x ±2%)

2.18 ms (256x ±2%)

2.18 ms (256x ±2%)

whatwg spec (235 kB)

5.54 µs

102 µs (18.5x ±2%)

82.7 µs (15.0x ±2%)

70.9 µs (12.9x ±9%)

144 µs (26.1x ±3%)

2.53 ms (458x ±2%)

3.13 ms (565x ±2%)

3.21 ms (581x ±3%)

Per-element matching runs each anchor on the page through a compiled div a[href] matcher – the shape a soupsieve port hits through turbohtml.query and its Matcher.match – raced against selectolax’s node match, soupsieve, BeautifulSoup, and pyquery. turbohtml answers each test with the same interned-atom comparison its select uses, walking the ancestor chain once per candidate, where the others re-interpret the parsed selector per element, so the sweep runs 34 to 44 times faster than selectolax, 75 to 145 times faster than soupsieve, and over a hundred times faster than BeautifulSoup and pyquery.

match each anchor against div a[href]

turbohtml

selectolax

soupsieve

BeautifulSoup (html.parser)

BeautifulSoup (lxml)

pyquery

daring fireball (10 kB)

1.85 µs

62.4 µs (33.8x ±3%)

139 µs (75.4x ±3%)

217 µs (118x ±3%)

223 µs (121x ±3%)

431 µs (234x ±3%)

ars technica (56 kB)

4.23 µs

152 µs (36.0x ±2%)

397 µs (93.9x ±2%)

582 µs (138x ±2%)

603 µs (143x ±2%)

1.02 ms (242x ±2%)

mozilla blog (95 kB)

5.83 µs

213 µs (36.6x ±1%)

607 µs (105x ±1%)

870 µs (150x ±1%)

901 µs (155x ±1%)

1.41 ms (243x ±1%)

whatwg spec (235 kB)

6.46 µs

287 µs (44.5x ±2%)

940 µs (146x ±3%)

1.24 ms (193x ±3%)

1.24 ms (192x ±2%)

1.66 ms (258x ±2%)

turbohtml.query.escape_identifier() escapes a raw string into a CSS identifier per the CSSOM serialize-an-identifier rules – the safe way to drop an untrusted class or id into a selector – against soupsieve’s escape. The workload escapes a thousand identifiers spanning leading digits, embedded specials, astral characters, and a lone dash. turbohtml walks the code points and emits the escape in C, where soupsieve builds the result through a per-character Python loop, so it runs 16 times faster.

escape 1,000 raw CSS identifiers

turbohtml

soupsieve

mixed shapes (1,000)

66.9 µs

1.08 ms (16.2x ±2%)

A text-content search runs through find_all() with text= (a regex matched against each element’s collected subtree text), raced against BeautifulSoup.find_all(string=...) and the equivalent text filters on lxml, parsel, and pyquery. When the text= filter is a plain string or a literal (no regex metacharacters, case-sensitive) compiled pattern, turbohtml gathers each candidate’s collected text and matches it in C – no Python str built, no per-element re.search call – where the others walk the tree in Python, so it leads every competitor by 1.2 to 3.0 times across these pages. A case-insensitive or otherwise non-literal pattern keeps the per-element Python path.

find by text content

turbohtml

lxml

pyquery

parsel

BeautifulSoup (html.parser)

BeautifulSoup (lxml)

daring fireball (10 kB)

24.4 µs

44 µs (1.9x ±3%)

50.6 µs (2.1x ±2%)

45.3 µs (1.9x ±1%)

55.4 µs (2.3x ±2%)

55.2 µs (2.3x ±2%)

ars technica (56 kB)

173 µs

239 µs (1.4x ±1%)

236 µs (1.4x ±1%)

240 µs (1.4x ±1%)

217 µs (1.3x ±1%)

218 µs (1.3x ±2%)

mozilla blog (95 kB)

278 µs

450 µs (1.7x ±2%)

441 µs (1.6x ±1%)

455 µs (1.7x ±1%)

465 µs (1.7x ±1%)

465 µs (1.7x ±2%)

whatwg spec (235 kB)

556 µs

1.24 ms (2.3x ±1%)

1.17 ms (2.1x ±1%)

1.26 ms (2.3x ±1%)

1.67 ms (3.1x ±2%)

1.68 ms (3.1x ±2%)

turbohtml.convert.css_specificity() weighs a selector list’s (a, b, c) specificity, raced against cssselect’s Selector.specificity(), the computation lxml, parsel, and pyquery inherit. turbohtml parses the selector and sums the weights in one C pass, so it leads across the type, compound, structural, complex, and grouped selectors below; cssselect parses in Python and builds a tree of selector objects first.

CSS selector specificity

turbohtml

cssselect

type

199 ns

493 ns (2.5x ±4%)

compound

342 ns

14.6 µs (42.6x ±7%)

structural

278 ns

13.2 µs (47.6x ±6%)

complex

528 ns

26 µs (49.3x ±9%)

group

549 ns

16.1 µs (29.4x ±9%)

XPath 1.0 evaluation runs through xpath(), raced against lxml’s libxml2 engine and parsel’s wrapper of it (selectolax and BeautifulSoup have no XPath). One expression per feature class (name tests, the // abbreviation, attribute, positional, and arithmetic predicates, string and aggregate functions, a reverse axis, a union, and a computed name test) runs over the 9.6 kB wpt page below; tox -e bench xpath repeats the sweep across every page size. turbohtml compiles each expression against the tree once, resolves name tests to interned atoms, and folds // to a single descendant walk, so it leads across the surface. The exception is a predicate that references position() ([1] or position() <= 3): it pins the result to proximity order and disables the // collapse, so on the largest pages lxml’s streaming evaluation closes the gap. Five rows exercise XPath 2.0 functions – ends-with, matches, replace, lower-case, and string-join – that turbohtml answers but libxml2 does not implement, so lxml and parsel show a gap there. Further rows are the lxml/parsel options the parity work added: a $variable binding, an EXSLT re:test predicate (turbohtml’s Python re against lxml’s C libexslt), an EXSLT set:distinct node-set reduction (built-in C dispatch on both sides, so it races C against C), a smart_strings attribute read, a custom extensions= function, an extensions= function whose return becomes a node-set feeding a later /@href step, a namespaces= prefix binding that resolves //svg:rect against {"svg": ".../2000/svg"} over a page carrying an SVG block, and a node-set $variable bound from a prior result ($rows/div, with rows reused from an earlier //div query) fed into a later path step. turbohtml still leads, since lxml resolves the namespace map and option set on every call. The last row precompiles the expression once with XPath and re-evaluates it, lxml’s etree.XPath doing the same: both skip the per-call parse xpath() pays, and turbohtml’s compiled program stays ahead per evaluation.

XPath feature surface (9.6 kB)

turbohtml

lxml

parsel

//div

2.37 µs

16.1 µs (6.8x ±5%)

50.4 µs (21.3x ±4%)

//a[@href]

504 ns

5.46 µs (10.9x ±9%)

8.14 µs (16.2x ±5%)

//div//a[@href]

2.13 µs

13.9 µs (6.6x ±5%)

16.8 µs (8.0x ±5%)

/html/body/div

1.06 µs

8.79 µs (8.4x ±7%)

26.1 µs (24.7x ±6%)

//div//a[1]

13.3 µs

14 µs (1.1x ±5%)

16.7 µs (1.3x ±5%)

//a[contains(@href, '/')]

496 ns

5.81 µs (11.8x ±7%)

8.49 µs (17.2x ±5%)

//div[position() <= 3]

7.84 µs

19.1 µs (2.5x ±4%)

40.6 µs (5.2x ±6%)

//a/ancestor::div

529 ns

3.31 µs (6.3x ±4%)

5.88 µs (11.2x ±4%)

//a | //span

835 ns

4.41 µs (5.3x ±7%)

6.92 µs (8.3x ±6%)

//*[local-name() = 'a']

6.36 µs

19.8 µs (3.2x ±5%)

22.8 µs (3.6x ±4%)

count(//a)

532 ns

3.44 µs (6.5x ±4%)

7.57 µs (14.3x ±5%)

//a[@href=$x] (variable)

681 ns

5.9 µs (8.7x ±5%)

8.45 µs (12.5x ±6%)

//a[re:test(@href, …)] (EXSLT)

501 ns

7.59 µs (15.2x ±6%)

8.56 µs (17.1x ±5%)

//a[ends-with(@href, …)] (XPath 2.0)

506 ns

—1

—1

string-join(//a/@href, …) (XPath 2.0)

669 ns

—2

—2

//a[lower-case(@href) = …] (XPath 2.0)

503 ns

—3

—3

//a[matches(@href, …)] (XPath 2.0)

492 ns

—4

—4

replace(//a/@href, …) (XPath 2.0)

1.33 µs

—5

—5

set:distinct(//a) (EXSLT)

588 ns

5.31 µs (9.1x ±3%)

6.11 µs (10.5x ±5%)

//a/@href (smart_strings)

658 ns

3.3 µs (5.1x ±5%)

5.67 µs (8.7x ±7%)

ext(//a) (extensions)

1.28 µs

4.11 µs (3.3x ±4%)

8.23 µs (6.5x ±5%)

ext(//a)/@href (node-set extension)

1.3 µs

4.28 µs (3.3x ±5%)

6.89 µs (5.3x ±5%)

//svg:rect (namespaces=)

792 ns

3.82 µs (4.9x ±6%)

6.3 µs (8.0x ±5%)

$rows/div (node-set variable)

3.5 µs

6.7 µs (2.0x ±5%)

26.3 µs (7.5x ±6%)

//a[@href] (precompiled, reused)

435 ns

3.61 µs (8.3x ±5%)

3.8 µs (8.8x ±5%)

1 ‘ends-with’

2 ‘string-join’

3 ‘lower-case’

4 ‘matches’

5 ‘replace’

XSLT

turbohtml.transform.Transform compiles one stylesheet into a native model with reusable XPath programs. Each application allocates source-specific indexes and output state. Callers can use one Transform instance with different documents and parameters across threads.

The first table measures construction. The other two measure a 120-row catalog and ten calls to a 300-template stylesheet. That stylesheet has 299 unused templates and 24 static xsl:number patterns in its used template; the repeated result includes any stylesheet analysis or XPath compilation left in the application path.

compile an XSLT stylesheet with 300 templates

turbohtml

lxml.etree

300 templates

116 µs

469 µs (4.1x ±3%)

XSLT transform a catalog (120 rows)

turbohtml

lxml.etree

catalog (120 rows)

175 µs

263 µs (1.6x ±2%)

apply one compiled 300-template stylesheet ten times

turbohtml

lxml.etree

300 templates

40.9 µs

81.1 µs (2.0x ±4%)

Node paths

turbohtml.Element.css_path() and xpath_path() return the unique locator that re-finds an element from the document root – a CSS selector and a positional XPath – against lxml’s getroottree().getpath(), the libxml2 path builder devtools’ “copy selector” mirrors, and (for the positional path) parsel’s wrapper of it. Each timed call walks every element in a pre-parsed page and serializes its path. Both methods lead getpath by roughly six times across these pages, narrowing to under threefold on the spec. css_path() previously rescanned the whole document to test each element’s id uniqueness, an O(N2) cost over a page that made it slower than getpath on id-heavy pages; a cached per-tree id-occurrence map (dropped with the element index on any mutation) now answers that test in O(1), so css_path keeps pace with the positional xpath_path.

node path for every element

turbohtml css_path

turbohtml xpath_path

lxml getpath

daring fireball (10 kB)

22.9 µs

24.1 µs (1.1x ±5%)

143 µs (6.3x ±3%)

ars technica (56 kB)

120 µs

124 µs (1.1x ±9%)

760 µs (6.4x ±7%)

mozilla blog (95 kB)

308 µs

345 µs (1.2x ±4%)

1.94 ms (6.3x ±4%)

whatwg spec (235 kB)

2.91 ms

2.96 ms (1.1x ±7%)

8.96 ms (3.1x ±6%)

Text content

The text suite collects the visible text two ways. First, the raw text join off a pre-parsed tree, the get_text pass: turbohtml’s text property concatenates every descendant text run, against lxml’s text_content(), resiliparse’s node text, selectolax’s text(), BeautifulSoup’s get_text(), and parsel’s and pyquery’s text extraction. turbohtml gathers the runs in one C walk into a buffer reserved up front, so it stays level with lxml and resiliparse, leads selectolax by 6.6 to 9.3 times and BeautifulSoup by 5.5 to 7.5, and runs 99 to 145 times ahead of parsel, which boxes each match in a wrapper first. pyquery trails by 33 to 57 times.

collect visible text

turbohtml

resiliparse1

lxml

BeautifulSoup (lxml)

BeautifulSoup (html.parser)

selectolax

pyquery

parsel

daring fireball (10 kB)

2.63 µs

2.48 µs (1.0x ±2%)

3.43 µs (1.4x ±2%)

18.7 µs (7.2x ±2%)

18.7 µs (7.2x ±2%)

21.4 µs (8.2x ±2%)

127 µs (48.4x ±1%)

296 µs (113x ±2%)

ars technica (56 kB)

13.5 µs

15.6 µs (1.2x ±3%)

15.2 µs (1.2x ±2%)

75.5 µs (5.6x ±1%)

74.8 µs (5.6x ±1%)

89.8 µs (6.7x ±1%)

448 µs (33.2x ±1%)

1.33 ms (98.6x ±1%)

mozilla blog (95 kB)

21.9 µs

30.1 µs (1.4x ±3%)

25.4 µs (1.2x ±1%)

164 µs (7.6x ±1%)

165 µs (7.6x ±2%)

203 µs (9.3x ±2%)

906 µs (41.5x ±1%)

3.18 ms (146x ±2%)

whatwg spec (235 kB)

83.7 µs

97.3 µs (1.2x ±3%)

87.5 µs (1.1x ±2%)

576 µs (6.9x ±1%)

585 µs (7.0x ±1%)

756 µs (9.1x ±1%)

4.8 ms (57.4x ±1%)

12.1 ms (145x ±2%)

1 resiliparse reports about 11% fewer elements than every other parser here (876 against 989 on the mozilla page), so it collects text from a smaller tree

Second, the layout-aware string-to-text extraction: turbohtml.Node.to_text() against inscriptis, the layout-aware HTML-to-text renderer it succeeds, html-text, Zyte’s plainer visible-text extractor, and resiliparse’s extract_plain_text. inscriptis and html-text both build an lxml tree in Python and resiliparse renders text off the lexbor tree it parses to, where turbohtml does the whole layout in one C walk; inscriptis additionally lays tables out as aligned columns, which html-text and resiliparse skip. turbohtml leads resiliparse by 2.4 to 4.1 times, html-text by 12 to 18 times, and inscriptis by 34 to 47 times.

layout-aware text

turbohtml

resiliparse

html-text

inscriptis

article (2 KiB)

7.41 µs

30.6 µs (4.2x ±4%)

135 µs (18.3x ±4%)

248 µs (33.5x ±5%)

table (4 KiB)

29.7 µs

72.2 µs (2.5x ±8%)

347 µs (11.7x ±7%)

1.38 ms (46.6x ±5%)

collapsed (2 KiB)

8.54 µs

—1

144 µs (16.9x ±7%)

—1

main (4 KiB)

7.36 µs

28.6 µs (3.9x ±6%)

—1

—1

annotated (4 KiB)

10.7 µs

—1

—1

314 µs (29.5x ±5%)

1 no equivalent operation

The collapsed row turns layout guessing off: turbohtml joins the stripped_strings word stream against html-text’s extract_text(guess_layout=False), 17 times faster; inscriptis and resiliparse have no comparable collapsed mode. The main row strips page boilerplate first, main_text() against resiliparse’s extract_plain_text(main_content=True), four times faster. The annotated row labels matching elements with spans through to_annotated_text() against inscriptis’s get_annotated_text, 75 times faster; html-text and resiliparse have no annotation surface, so they sit out that row.

Tree navigation

Walking every descendant of a parsed tree: turbohtml’s descendants iterator against resiliparse’s node walk, BeautifulSoup’s descendants, lxml’s iterdescendants(), selectolax’s node iteration, pyquery, and html5lib. The list(el), iterdescendants(), and iterancestors() family ports to children, descendants, and ancestors; the descendant walk is the dominant case. Each timed call consumes the whole iterator, where turbohtml yields interned nodes straight from the arena faster than lxml’s libxml2 proxy objects and BeautifulSoup’s Python NavigableString chain. resiliparse’s lexbor walk stays closest at 1.4 to 2.0 times and BeautifulSoup’s descendants – one of its leaner paths – at about twice, while lxml and selectolax trail by five times, pyquery by twenty, and html5lib by ninety.

walk every descendant

turbohtml

resiliparse1

BeautifulSoup (html.parser)

BeautifulSoup (lxml)

lxml

selectolax

pyquery

html5lib

daring fireball (10 kB)

3.37 µs

4.83 µs (1.5x ±6%)

7.01 µs (2.1x ±14%)

7.45 µs (2.3x ±22%)

17.2 µs (5.1x ±2%)

26.3 µs (7.8x ±36%)

75.7 µs (22.5x ±1%)

281 µs (83.5x ±1%)

ars technica (56 kB)

13.3 µs

21.1 µs (1.6x ±12%)

26.2 µs (2.0x ±2%)

26.2 µs (2.0x ±1%)

64.1 µs (4.9x ±1%)

69.1 µs (5.2x ±4%)

278 µs (20.9x ±3%)

1.17 ms (88.0x ±1%)

mozilla blog (95 kB)

28.5 µs

57 µs (2.0x ±31%)

56.1 µs (2.0x ±4%)

54 µs (1.9x ±1%)

135 µs (4.8x ±1%)

152 µs (5.4x ±1%)

616 µs (21.6x ±3%)

2.58 ms (90.4x ±2%)

whatwg spec (235 kB)

97.5 µs

171 µs (1.8x ±2%)

192 µs (2.0x ±3%)

198 µs (2.1x ±13%)

473 µs (4.9x ±1%)

518 µs (5.4x ±2%)

2.13 ms (21.9x ±1%)

8.99 ms (92.3x ±1%)

1 resiliparse reports about 11% fewer elements than every other parser here (876 against 989 on the mozilla page), so it collects text from a smaller tree

Serializing

Serializing a parsed document back to HTML: turbohtml’s html against resiliparse’s, pyquery’s, selectolax’s, parsel’s, and lxml’s serializers, BeautifulSoup’s decode, and html5lib. turbohtml scans each text run for the next character that needs escaping (two code points at a time with the same SWAR lane probes escape() uses) and bulk-copies the clean spans, recovering each special’s position from the lane mask, and reserves the whole-document buffer up front so the output grows in one allocation. It serializes about twice as fast as resiliparse and pyquery, four to six times faster than selectolax, parsel, and lxml, and 52 to 77 times faster than BeautifulSoup and html5lib.

serialize a parsed tree

turbohtml

resiliparse

pyquery

selectolax

parsel

lxml

BeautifulSoup (lxml)

BeautifulSoup (html.parser)

html5lib

daring fireball (10 kB)

6.52 µs

13 µs (2.0x ±2%)

17.2 µs (2.7x ±2%)

28.6 µs (4.4x ±2%)

34 µs (5.3x ±3%)

39.7 µs (6.1x ±2%)

409 µs (62.7x ±2%)

413 µs (63.4x ±2%)

468 µs (71.8x ±2%)

ars technica (56 kB)

34.8 µs

72.9 µs (2.1x ±1%)

76.7 µs (2.3x ±2%)

154 µs (4.5x ±1%)

168 µs (4.9x ±1%)

195 µs (5.7x ±1%)

1.78 ms (51.1x ±1%)

1.8 ms (51.8x ±1%)

1.89 ms (54.3x ±1%)

mozilla blog (95 kB)

67.4 µs

148 µs (2.2x ±1%)

155 µs (2.3x ±3%)

307 µs (4.6x ±1%)

366 µs (5.5x ±1%)

423 µs (6.3x ±2%)

4.09 ms (60.7x ±1%)

4.14 ms (61.5x ±1%)

4.11 ms (61.0x ±1%)

whatwg spec (235 kB)

174 µs

363 µs (2.1x ±1%)

377 µs (2.2x ±2%)

739 µs (4.3x ±1%)

687 µs (4.0x ±2%)

808 µs (4.7x ±2%)

10.8 ms (62.2x ±1%)

11 ms (62.9x ±1%)

13.5 ms (77.5x ±1%)

Minifying

Minifying a document with turbohtml.clean.minify(): parse, then serialize once with every fold engaged (collapsing insignificant whitespace, omitting the WHATWG-optional tags, unquoting attributes, and stripping comments), against minify-html’s Rust minifier on the same folds (its CSS and JS minification left off for a like-for-like comparison), the pure-Python htmlmin and css-html-js-minify, and the native CLI minifiers html-minifier-terser and tdewolff/minify. turbohtml parses and emits in C through one preallocated buffer, so with the parse included it runs roughly two to three times faster than minify-html and sixteen to seventy times faster than the pure-Python pair. html-minifier-terser and tdewolff are invoked through their command line, so their millisecond timings are dominated by process startup; the clean comparison against them is output size. turbohtml lands within about two percent of html-minifier-terser on the structural folds both apply; minify-html folds more aggressively for roughly three to ten percent smaller, and tdewolff goes further still by also minifying the inline CSS and JavaScript turbohtml leaves untouched here.

minify a document

turbohtml

tdewolff

minify-html

css-html-js-minify

htmlmin

html-minifier-terser

size

time

size

time

size

time

size

time

size

time

size

time

daring fireball (10 kB)

8.48 kB

27.4 µs

6.42 kB (0.76x)

4.5 ms (165x ±7%)

7.94 kB (0.94x)

67.6 µs (2.5x ±3%)

7.39 kB (0.87x)

636 µs (23.3x ±1%)

8.55 kB (1.01x)

592 µs (21.6x ±1%)

8.32 kB (0.98x)

72.6 ms (2651x ±1%)

ars technica (56 kB)

38.7 kB

127 µs

—1

—1

37 kB (0.95x)

319 µs (2.6x ±3%)

38.5 kB (0.99x)

5.38 ms (42.5x ±1%)

38.2 kB (0.99x)

2.68 ms (21.2x ±1%)

38 kB (0.98x)

78.3 ms (619x ±1%)

mozilla blog (95 kB)

54.2 kB

263 µs

48.7 kB (0.90x)

5.03 ms (19.2x ±1%)

48.9 kB (0.90x)

749 µs (2.9x ±1%)

53.3 kB (0.98x)

17.7 ms (67.3x ±1%)

54.4 kB (1.003x)

6.05 ms (23.0x ±2%)

53.6 kB (0.99x)

83.4 ms (317x ±1%)

whatwg spec (235 kB)

209 kB

919 µs

199 kB (0.95x)

6.67 ms (7.3x ±2%)

203 kB (0.97x)

1.71 ms (1.9x ±3%)

227 kB (1.09x)

36.2 ms (39.4x ±2%)

210 kB (1.003x)

15 ms (16.4x ±2%)

210 kB (1.01x)

90.5 ms (98.5x ±2%)

1 cannot minify -: unexpected < in expression on line 109 and column 17

Building

The write path: construct a <ul> of N <li> rows from scratch (each with a class, a data attribute, and a text child), then serialize it, the work an editor or template engine does. turbohtml’s arena allocation and interned attribute names make construction cheaper than lxml’s libxml2 nodes and far cheaper than BeautifulSoup’s Python objects. selectolax is parse-only, so it has no entry.

build a list (constructors)

turbohtml

lxml

BeautifulSoup (html.parser)

100 rows

57.1 µs

137 µs (2.5x ±4%)

782 µs (13.7x ±4%)

1k rows

544 µs

1.35 ms (2.5x ±3%)

7.47 ms (13.8x ±4%)

10k rows

5.41 ms

13.4 ms (2.5x ±3%)

82.2 ms (15.2x ±10%)

The construct and emit commands split that aggregate over the same builders: construct builds the rows and stops before serialization, and emit serializes a tree built once outside the timed region. On construct turbohtml’s arena keeps it roughly twice as fast as lxml and ahead of most Python builders, though the leanest string builders markyp and simple-html edge it out; on emit its SWAR serializer pulls ahead of every alternative, from 1.6 times over simple-html to eight times over lxml and ninety times over BeautifulSoup.

construct N elements (no serialize)

turbohtml

markyp1

simple-html2

lxml

htbuilder3

hyperpython

htpy4

dominate

BeautifulSoup (html.parser)

100 rows

42.9 µs

22.8 µs (0.6x ±4%)

35.8 µs (0.9x ±5%)

96 µs (2.3x ±5%)

152 µs (3.6x ±5%)

191 µs (4.5x ±3%)

239 µs (5.6x ±3%)

254 µs (6.0x ±5%)

261 µs (6.1x ±3%)

1k rows

436 µs

229 µs (0.6x ±3%)

367 µs (0.9x ±3%)

955 µs (2.2x ±3%)

1.54 ms (3.6x ±6%)

1.91 ms (4.4x ±3%)

2.47 ms (5.7x ±5%)

2.45 ms (5.7x ±3%)

2.55 ms (5.9x ±4%)

10k rows

4.34 ms

2.29 ms (0.6x ±4%)

3.52 ms (0.9x ±5%)

9.76 ms (2.3x ±4%)

15.2 ms (3.5x ±4%)

19.3 ms (4.5x ±5%)

24.5 ms (5.7x ±5%)

25.1 ms (5.8x ±6%)

25.4 ms (5.9x ±4%)

1 markyp concatenates a string rather than constructing a navigable tree, so nothing it builds can afterwards be queried or mutated

2 simple-html concatenates a string rather than constructing a navigable tree, so nothing it builds can afterwards be queried or mutated

3 htbuilder concatenates a string rather than constructing a navigable tree, so nothing it builds can afterwards be queried or mutated

4 htpy concatenates a string rather than constructing a navigable tree, so nothing it builds can afterwards be queried or mutated

emit a built tree

turbohtml

simple-html

lxml

htbuilder

htpy

markyp

dominate

hyperpython

BeautifulSoup (html.parser)

100 rows

4.33 µs

8.37 µs (2.0x ±2%)

35.6 µs (8.3x ±2%)

41.8 µs (9.7x ±2%)

80.8 µs (18.7x ±2%)

84 µs (19.5x ±2%)

98 µs (22.7x ±4%)

134 µs (31.0x ±2%)

389 µs (89.9x ±3%)

1k rows

43 µs

80 µs (1.9x ±2%)

347 µs (8.1x ±2%)

420 µs (9.8x ±3%)

789 µs (18.4x ±1%)

816 µs (19.0x ±1%)

920 µs (21.5x ±1%)

1.35 ms (31.5x ±11%)

3.85 ms (89.6x ±1%)

10k rows

453 µs

857 µs (1.9x ±21%)

3.51 ms (7.8x ±5%)

4.12 ms (9.1x ±3%)

7.8 ms (17.3x ±3%)

8.15 ms (18.0x ±3%)

9.15 ms (20.3x ±3%)

14.1 ms (31.1x ±3%)

40.9 ms (90.3x ±5%)

The terse turbohtml.build.E builder spells the same <ul> declaratively, raced against ten dedicated HTML generators. The leanest string builders simple-html and markyp build it faster (0.4x and roughly 0.9x), yattag runs on par, and E leads the rest – lxml.builder, htbuilder, fast-html, hyperpython, htpy, dominate, and airium – by 1.5 to 7 times, and unlike any of them it returns a real, queryable turbohtml tree rather than a string. That tree costs a little over twice the raw Element constructor above – the price of the leading-mapping and per-child dispatch the sugar runs in Python.

build a list (terse builders)

turbohtml

simple-html1

markyp2

yattag3

lxml.builder

htbuilder4

fast-html5

hyperpython

htpy6

dominate

airium

100 rows

114 µs

44 µs (0.4x ±2%)

104 µs (1.0x ±4%)

144 µs (1.3x ±4%)

198 µs (1.8x ±2%)

219 µs (2.0x ±2%)

279 µs (2.5x ±1%)

335 µs (3.0x ±2%)

353 µs (3.1x ±1%)

440 µs (3.9x ±2%)

756 µs (6.7x ±2%)

1k rows

1.17 ms

450 µs (0.4x ±3%)

1 ms (0.9x ±2%)

1.39 ms (1.2x ±4%)

2 ms (1.8x ±3%)

2.39 ms (2.1x ±3%)

2.77 ms (2.4x ±2%)

3.42 ms (3.0x ±2%)

3.46 ms (3.0x ±2%)

4.37 ms (3.8x ±4%)

7.32 ms (6.3x ±3%)

10k rows

13.7 ms

4.83 ms (0.4x ±2%)

10.9 ms (0.8x ±4%)

13.4 ms (1.0x ±2%)

20.3 ms (1.5x ±2%)

25.9 ms (1.9x ±2%)

28.7 ms (2.1x ±3%)

34.7 ms (2.6x ±2%)

35.1 ms (2.6x ±2%)

46.6 ms (3.5x ±8%)

75.8 ms (5.6x ±4%)

1 simple-html concatenates a string rather than constructing a navigable tree, so nothing it builds can afterwards be queried or mutated

2 markyp concatenates a string rather than constructing a navigable tree, so nothing it builds can afterwards be queried or mutated

3 yattag concatenates a string rather than constructing a navigable tree, so nothing it builds can afterwards be queried or mutated

4 htbuilder concatenates a string rather than constructing a navigable tree, so nothing it builds can afterwards be queried or mutated

5 fast-html concatenates a string rather than constructing a navigable tree, so nothing it builds can afterwards be queried or mutated

6 htpy concatenates a string rather than constructing a navigable tree, so nothing it builds can afterwards be queried or mutated

Editing

Editing a parsed tree: tag every <a> with rel="nofollow", a link-rewriting pass. Because the pass mutates the tree, each library rebuilds a fresh parse before every iteration outside the timed region, then the timed call walks its links and sets the attribute (turbohtml through the live attrs mapping, resiliparse and selectolax through their node setters, lxml through Element.set, pyquery through its attr, BeautifulSoup through item assignment). turbohtml leads resiliparse by 1.4 to 2.1 times, lxml by 1.7 to 3.7, selectolax by 3.0 to 4.9, and pyquery and BeautifulSoup by 2.8 to 12 times, the gap widening with the page as each reparse costs more.

tag every link rel=nofollow

turbohtml

resiliparse

lxml

selectolax

BeautifulSoup (lxml)

BeautifulSoup (html.parser)

pyquery

daring fireball (10 kB)

4.74 µs

7.14 µs (1.6x ±4%)

10.9 µs (2.3x ±3%)

16 µs (3.4x ±3%)

26.2 µs (5.6x ±3%)

33.1 µs (7.0x ±6%)

37.9 µs (8.1x ±8%)

ars technica (56 kB)

16.6 µs

17.8 µs (1.1x ±4%)

27.7 µs (1.7x ±4%)

39.1 µs (2.4x ±4%)

75.6 µs (4.6x ±3%)

61.5 µs (3.8x ±5%)

122 µs (7.4x ±25%)

mozilla blog (95 kB)

25.4 µs

29.6 µs (1.2x ±6%)

58.7 µs (2.4x ±35%)

63.6 µs (2.6x ±5%)

130 µs (5.2x ±2%)

122 µs (4.8x ±3%)

221 µs (8.8x ±16%)

whatwg spec (235 kB)

51.4 µs

62.5 µs (1.3x ±9%)

83.3 µs (1.7x ±13%)

118 µs (2.4x ±2%)

402 µs (7.9x ±3%)

456 µs (8.9x ±12%)

388 µs (7.6x ±6%)

A second pass churns the class list: add then drop a token on every link (turbohtml’s add_class()/remove_class() against resiliparse’s, selectolax’s, and pyquery’s class edits, lxml’s classes set, and BeautifulSoup’s attribute assignment). The add-then-remove is a net no-op, so each repeat does equal work. turbohtml leads resiliparse by three to six times, selectolax and lxml by roughly ten to eighteen times, and BeautifulSoup by up to thirty-seven times.

class add/remove on every link

turbohtml

resiliparse

selectolax

pyquery

lxml

BeautifulSoup (html.parser)

BeautifulSoup (lxml)

daring fireball (10 kB)

2.62 µs

8.73 µs (3.4x ±2%)

26.6 µs (10.2x ±2%)

41.5 µs (15.9x ±2%)

45.7 µs (17.5x ±2%)

21.1 µs (8.1x ±1%)

30 µs (11.5x ±1%)

ars technica (56 kB)

9.67 µs

25.6 µs (2.7x ±3%)

73.2 µs (7.6x ±2%)

108 µs (11.2x ±2%)

124 µs (12.9x ±2%)

68 µs (7.1x ±1%)

90.5 µs (9.4x ±3%)

mozilla blog (95 kB)

10.3 µs

39.1 µs (3.9x ±3%)

102 µs (10.0x ±2%)

137 µs (13.5x ±2%)

161 µs (15.7x ±3%)

125 µs (12.2x ±2%)

154 µs (15.0x ±1%)

whatwg spec (235 kB)

10.1 µs

64.7 µs (6.5x ±2%)

156 µs (15.5x ±2%)

178 µs (17.7x ±2%)

190 µs (19.0x ±2%)

388 µs (38.6x ±6%)

414 µs (41.2x ±5%)

Two content setters replace the body’s children on a freshly parsed tree. set_inner_html() reparses a fixed fragment in the <body>’s context and splices it in one C call, against lxml clearing the body and appending fragments_fromstring, pyquery’s .html(), and BeautifulSoup clearing it and appending a reparsed soup; it leads them by 6.6 to 9.4, 1.7 to 11, and 15 to 42 times. set_text() replaces the children with one verbatim text node, against the same three, leading by 6.8 to 9.9, 2.3 to 14, and 11 to 22 times. BeautifulSoup trails furthest because it reparses the whole page on every iteration where the others splice into a live tree.

replace body inner HTML

turbohtml

lxml

pyquery

BeautifulSoup (lxml)

BeautifulSoup (html.parser)

daring fireball (10 kB)

2.12 µs

13.9 µs (6.6x ±2%)

17.5 µs (8.3x ±2%)

89.6 µs (42.4x ±2%)

88.2 µs (41.7x ±3%)

ars technica (56 kB)

7.8 µs

47.7 µs (6.2x ±6%)

57.4 µs (7.4x ±6%)

137 µs (17.6x ±7%)

142 µs (18.2x ±8%)

mozilla blog (95 kB)

11.7 µs

108 µs (9.3x ±2%)

123 µs (10.6x ±2%)

203 µs (17.4x ±4%)

208 µs (17.8x ±7%)

whatwg spec (235 kB)

35.3 µs

259 µs (7.4x ±2%)

363 µs (10.3x ±2%)

807 µs (22.9x ±4%)

804 µs (22.8x ±4%)

replace body text

turbohtml

lxml

pyquery

BeautifulSoup (html.parser)

BeautifulSoup (lxml)

daring fireball (10 kB)

1.13 µs

9.08 µs (8.1x ±2%)

14.7 µs (13.0x ±2%)

23.7 µs (20.9x ±1%)

23.6 µs (20.9x ±2%)

ars technica (56 kB)

6.4 µs

43 µs (6.8x ±2%)

54.1 µs (8.5x ±1%)

70.3 µs (11.0x ±1%)

79.9 µs (12.5x ±2%)

mozilla blog (95 kB)

10.5 µs

102 µs (9.8x ±2%)

118 µs (11.3x ±1%)

121 µs (11.6x ±4%)

116 µs (11.1x ±1%)

whatwg spec (235 kB)

33 µs

254 µs (7.7x ±1%)

357 µs (10.9x ±1%)

709 µs (21.5x ±3%)

695 µs (21.1x ±1%)

A bulk tag edit over each page’s <code>/<a>/<q> elements: remove() drops each match with its subtree, and strip_tags() unwraps each match but keeps its content. Both rewrites are destructive, so the timed call parses the page afresh – the string-to-result transform these helpers perform – and races each library’s own bulk tag helper: w3lib’s regex remove_tags, resiliparse, lxml, pyquery, selectolax, and BeautifulSoup. On strip_tags turbohtml’s single C pass leads every alternative by roughly three to seven times, and BeautifulSoup by nearly sixty. On remove it leads the tree libraries by the same margin but trails w3lib’s pure-regex strip on the larger pages, where deleting whole subtrees by regex skips the per-node work a real tree edit does.

drop tags with content (remove)

turbohtml

w3lib1

resiliparse

lxml

pyquery

selectolax

BeautifulSoup (lxml)

BeautifulSoup (html.parser)

daring fireball (10 kB)

23.2 µs

23.8 µs (1.1x ±2%)

42 µs (1.9x ±1%)

82.8 µs (3.6x ±1%)

115 µs (5.0x ±6%)

98.5 µs (4.3x ±1%)

1.1 ms (47.5x ±1%)

1.32 ms (57.2x ±1%)

ars technica (56 kB)

112 µs

96.2 µs (0.9x ±1%)

183 µs (1.7x ±1%)

428 µs (3.9x ±2%)

453 µs (4.1x ±1%)

480 µs (4.3x ±3%)

4.82 ms (42.9x ±3%)

5.7 ms (50.7x ±2%)

mozilla blog (95 kB)

252 µs

153 µs (0.7x ±1%)

402 µs (1.6x ±2%)

974 µs (3.9x ±2%)

886 µs (3.6x ±1%)

1.53 ms (6.1x ±3%)

11.4 ms (45.4x ±4%)

13.6 ms (53.9x ±3%)

whatwg spec (235 kB)

650 µs

318 µs (0.5x ±2%)

1.06 ms (1.7x ±3%)

1.98 ms (3.1x ±2%)

2.2 ms (3.4x ±2%)

2.77 ms (4.3x ±3%)

31.6 ms (48.6x ±10%)

38.1 ms (58.6x ±10%)

1 w3lib removes the tags with a regular expression over the raw string and never parses, so it cannot honor nesting or the tokenizer rules that decide where an element really ends

unwrap tags keep content (strip_tags)

turbohtml

w3lib

lxml

pyquery

selectolax

BeautifulSoup (lxml)

BeautifulSoup (html.parser)

daring fireball (10 kB)

24.4 µs

68.4 µs (2.9x ±2%)

87.7 µs (3.6x ±6%)

132 µs (5.5x ±2%)

113 µs (4.7x ±2%)

1.25 ms (51.3x ±3%)

1.43 ms (58.5x ±2%)

ars technica (56 kB)

120 µs

319 µs (2.7x ±4%)

452 µs (3.8x ±2%)

525 µs (4.4x ±2%)

521 µs (4.4x ±1%)

5.74 ms (48.0x ±4%)

6.51 ms (54.4x ±3%)

mozilla blog (95 kB)

261 µs

630 µs (2.5x ±1%)

1.08 ms (4.2x ±10%)

973 µs (3.8x ±4%)

1.61 ms (6.2x ±2%)

12.3 ms (47.3x ±6%)

14.3 ms (54.8x ±2%)

whatwg spec (235 kB)

669 µs

1.64 ms (2.5x ±3%)

2.06 ms (3.1x ±2%)

2.59 ms (3.9x ±3%)

2.93 ms (4.4x ±2%)

33.1 ms (49.5x ±12%)

39.7 ms (59.4x ±9%)

Extraction

Pulling values out of a document, the idioms the parsel, pyquery, and w3lib migrations center on. First, reading every matched node’s @href and visible text off a pre-parsed page: turbohtml selects once and reads attr() and text off each node, against resiliparse, lxml, selectolax, parsel, and pyquery selecting and reading, and BeautifulSoup. turbohtml compiles the selector once and reads interned atoms, where the others re-translate the CSS per call or box every match in a wrapper object, so it leads resiliparse by two to six times, lxml and selectolax by five to seventeen times, parsel and pyquery by twenty to seventy times, and BeautifulSoup by up to 260 times.

extract @href per match

turbohtml

resiliparse

lxml

selectolax

parsel

pyquery

BeautifulSoup (html.parser)

BeautifulSoup (lxml)

daring fireball (10 kB)

3.57 µs

7.28 µs (2.1x ±5%)

17.9 µs (5.1x ±7%)

19 µs (5.4x ±7%)

95.8 µs (26.9x ±5%)

237 µs (66.6x ±5%)

133 µs (37.5x ±9%)

135 µs (38.0x ±6%)

ars technica (56 kB)

9.06 µs

22.5 µs (2.5x ±4%)

43.3 µs (4.8x ±4%)

66.5 µs (7.4x ±4%)

208 µs (23.1x ±5%)

542 µs (59.9x ±5%)

470 µs (51.9x ±4%)

485 µs (53.6x ±4%)

mozilla blog (95 kB)

12.5 µs

37 µs (3.0x ±5%)

64.4 µs (5.2x ±4%)

102 µs (8.3x ±4%)

299 µs (24.0x ±5%)

764 µs (61.3x ±4%)

1 ms (80.6x ±6%)

1.04 ms (83.3x ±4%)

whatwg spec (235 kB)

14.4 µs

71.5 µs (5.0x ±12%)

105 µs (7.3x ±12%)

149 µs (10.4x ±12%)

386 µs (26.8x ±13%)

939 µs (65.2x ±12%)

3.38 ms (235x ±13%)

3.39 ms (236x ±12%)

extract text per match

turbohtml

resiliparse

selectolax

lxml

parsel

pyquery

BeautifulSoup (lxml)

BeautifulSoup (html.parser)

daring fireball (10 kB)

3.44 µs

7.01 µs (2.1x ±6%)

22.7 µs (6.6x ±5%)

55.2 µs (16.1x ±6%)

94.8 µs (27.6x ±5%)

143 µs (41.6x ±12%)

155 µs (45.2x ±6%)

167 µs (48.6x ±12%)

ars technica (56 kB)

8.73 µs

22.6 µs (2.6x ±6%)

59.3 µs (6.8x ±5%)

119 µs (13.7x ±5%)

193 µs (22.2x ±8%)

339 µs (38.8x ±5%)

535 µs (61.3x ±7%)

577 µs (66.1x ±18%)

mozilla blog (95 kB)

14.6 µs

38.7 µs (2.7x ±15%)

104 µs (7.2x ±16%)

179 µs (12.3x ±17%)

286 µs (19.6x ±18%)

507 µs (34.7x ±15%)

1.14 ms (78.0x ±17%)

1.11 ms (76.3x ±17%)

whatwg spec (235 kB)

14.6 µs

72.1 µs (5.0x ±7%)

166 µs (11.4x ±7%)

226 µs (15.5x ±8%)

338 µs (23.1x ±9%)

598 µs (40.9x ±14%)

3.51 ms (241x ±13%)

3.44 ms (236x ±9%)

Second, reading a document’s own URL hints: turbohtml’s base_url() and meta_refresh() against w3lib’s get_base_url and get_meta_refresh and the same read off lxml, resiliparse, selectolax, parsel, pyquery, and BeautifulSoup trees. Every alternative parses the string each call; turbohtml runs the WHATWG tree builder and reads the hint off the parsed <head>, leading w3lib’s regular-expression pass by five to six times and the other parsers by four to forty-six times on this small document.

extract URL hints

turbohtml

lxml

resiliparse

w3lib

selectolax

parsel

pyquery

BeautifulSoup (html.parser)

BeautifulSoup (lxml)

base_url / get_base_url

1.17 µs

4.84 µs (4.2x ±3%)

5.07 µs (4.4x ±4%)

7.36 µs (6.4x ±2%)

8.43 µs (7.3x ±3%)

12.1 µs (10.4x ±2%)

13 µs (11.2x ±3%)

53 µs (45.5x ±3%)

52.9 µs (45.4x ±3%)

meta_refresh / get_meta_refresh

1.21 µs

5.42 µs (4.5x ±2%)

5.27 µs (4.4x ±3%)

6.5 µs (5.4x ±2%)

8.85 µs (7.4x ±2%)

12.8 µs (10.6x ±2%)

20.9 µs (17.3x ±3%)

55.3 µs (45.7x ±3%)

56 µs (46.2x ±2%)

Fluent chaining

A pyquery-style fluent chain over a pre-parsed tree: select every <a>, keep the linked ones, take the first, tag it, and read its href (turbohtml’s turbohtml.query.Query against pyquery, whose wrapper delegates to lxml). Both wrappers are thin Python over the underlying engine, so the gap is the engine’s: turbohtml’s selector and attribute primitives run in C, and the wrapper avoids a redundant de-duplication when the chain starts from one node, so it runs four to nearly fifty times faster, depending on how much the page exercises the selector.

fluent jQuery-style chain

turbohtml

pyquery

daring fireball (10 kB)

1.74 µs

82.6 µs (47.6x ±1%)

ars technica (56 kB)

5.34 µs

175 µs (32.9x ±1%)

mozilla blog (95 kB)

8.09 µs

238 µs (29.5x ±3%)

whatwg spec (235 kB)

17.3 µs

302 µs (17.5x ±3%)

html.parser adapter

turbohtml.migration.stdlib.HTMLParser against the standard library’s html.parser.HTMLParser and lxml’s target-parser API, all driven with the same minimal handler so the comparison is the parser and dispatch cost for the identical callback-driven programming model. The per-tag Python handler call is a floor both Python parsers pay. Dispatch runs in C: the tokenizer calls the handle_* methods itself, binding them once per feed rather than building a token object and walking its fields in Python for every token. Under Callgrind that removes 30.8% of the instructions the whole workload executes, and nearly half its indirect branches, which is what the interpreter spends on dispatch. turbohtml runs 7.5 to 11.2 times faster than html.parser and 1.7 to 2.7 times faster than lxml’s fully native target parser, which never crosses into Python per tag.

feed and dispatch a page

turbohtml

lxml

html.parser

daring fireball (10 kB)

98.9 µs

268 µs (2.8x ±23%)

1.1 ms (11.2x ±19%)

ars technica (56 kB)

494 µs

1.18 ms (2.4x ±14%)

3.89 ms (7.9x ±18%)

mozilla blog (95 kB)

1.24 ms

2.09 ms (1.7x ±52%)

9.34 ms (7.6x ±47%)

whatwg spec (235 kB)

2.61 ms

4.46 ms (1.8x ±16%)

24.7 ms (9.5x ±21%)

CSS minification

turbohtml.clean.minify_css() against the CSS minifiers on PyPI, over the unminified source CSS these frameworks publish. Each minifier column pairs its output size with the time to produce it; the ratio in each cell is against turbohtml. Sizes are deterministic byte counts; times are the minimum of repeated runs. esbuild and tdewolff/minify are native minifiers invoked through their command line, so their millisecond timings are dominated by process startup; against them the clean comparison is output size, where turbohtml stays within a couple percent and comes out smaller on most of the corpus.

minify CSS

turbohtml

lightningcss

rcssmin2

esbuild

tdewolff

csscompressor

cssmin3

css-html-js-minify4

size

time

size

time

size

time

size

time

size

time

size

time

size

time

size

time

normalize.css (6 kB)

1.75 kB

32.5 µs

1.75 kB (1.001x)

78.1 µs (2.5x ±15%)

1.77 kB (1.01x)

7.23 µs (0.3x ±12%)

1.82 kB (1.04x)

8.97 ms (277x ±14%)

1.81 kB (1.03x)

6.98 ms (215x ±11%)

1.84 kB (1.05x)

1.55 ms (47.9x ±10%)

1.85 kB (1.06x)

548 µs (16.9x ±11%)

1.83 kB (1.05x)

928 µs (28.6x ±14%)

pico.css (90 kB)

81.2 kB

1.08 ms

80 kB (0.99x)

2.27 ms (2.1x ±16%)

82.1 kB (1.01x)

262 µs (0.3x ±11%)

82.5 kB (1.02x)

15.6 ms (14.5x ±13%)

82 kB (1.01x)

8.98 ms (8.4x ±11%)

81.6 kB (1.01x)

47.3 ms (43.7x ±12%)

81.8 kB (1.01x)

291 ms (269x ±11%)

81.9 kB (1.01x)

310 ms (287x ±12%)

animate.css (93 kB)

73.8 kB

1.29 ms

68.8 kB (0.93x)

1.7 ms (1.4x ±8%)

75.7 kB (1.03x)

221 µs (0.2x ±7%)

73.5 kB (0.995x)

16.2 ms (12.6x ±10%)

74.4 kB (1.01x)

8.87 ms (6.9x ±9%)

75.7 kB (1.03x)

34.3 ms (26.6x ±8%)

75.8 kB (1.03x)

9.97 ms (7.8x ±9%)

75.8 kB (1.03x)

17.2 ms (13.4x ±16%)

foundation.css (164 kB)

133 kB

2.82 ms

—1

—1

137 kB (1.03x)

485 µs (0.2x ±8%)

132 kB (0.99x)

19 ms (6.8x ±13%)

136 kB (1.02x)

10.2 ms (3.7x ±9%)

136 kB (1.03x)

80.4 ms (28.6x ±9%)

136 kB (1.03x)

665 ms (237x ±8%)

137 kB (1.03x)

689 ms (245x ±9%)

bootstrap.css (274 kB)

230 kB

3.73 ms

229 kB (0.996x)

6.72 ms (1.9x ±15%)

233 kB (1.02x)

806 µs (0.3x ±13%)

233 kB (1.02x)

23 ms (6.2x ±12%)

232 kB (1.01x)

12.5 ms (3.4x ±11%)

234 kB (1.02x)

107 ms (28.8x ±13%)

232 kB (1.01x)

838 ms (225x ±14%)

234 kB (1.02x)

1.28 s (343x ±42%)

bulma.css (745 kB)

681 kB

8.69 ms

674 kB (0.99x)

17.2 ms (2.0x ±17%)

680 kB (0.998x)

2.3 ms (0.3x ±13%)

689 kB (1.01x)

42.4 ms (4.9x ±24%)

685 kB (1.01x)

18.9 ms (2.2x ±13%)

681 kB (1.000x)

772 ms (88.9x ±13%)

679 kB (0.997x)

3.9 s (449x ±13%)

681 kB (1.000x)

4.19 s (483x ±18%)

1 Parsing stylesheet failed: Invalid media query at :577:29

2 rcssmin strips whitespace and comments without parsing, so it applies none of the color, number, and shorthand rewrites turbohtml does; on these stylesheets that leaves its output within 1.01-1.03x of turbohtml’s, and 0.998x on bulma, so the structural shortenings change little on already-tight framework CSS

3 cssmin strips whitespace and comments without parsing, so it applies none of the color, number, and shorthand rewrites turbohtml does; on these stylesheets that leaves its output within 1.01-1.03x of turbohtml’s, and 0.998x on bulma, so the structural shortenings change little on already-tight framework CSS

4 css-html-js-minify strips whitespace and comments without parsing, so it applies none of the color, number, and shorthand rewrites turbohtml does; on these stylesheets that leaves its output within 1.01-1.03x of turbohtml’s, and 0.998x on bulma, so the structural shortenings change little on already-tight framework CSS

csscompressor (the YUI port) and cssmin (its BSD descendant) rewrite values to their shortest form the way turbohtml does, but as pure-Python regex passes they turn quadratic on a large stylesheet and trail the C engine by tens to over four hundred times, cssmin and css-html-js-minify reaching roughly four seconds on the 745 kB bulma.css where turbohtml takes 9 ms. rcssmin is a C extension and faster than turbohtml, though it only strips comments and whitespace, so it leaves a larger result everywhere except the custom-property-heavy bulma.css. css-html-js-minify is among the slowest of the set. The three pure-Python tools and rcssmin also break value safety: each rewrites the internal whitespace of a custom-property value, which CSS Variables 1 §2 keeps as the literal token stream that var() splices verbatim and getPropertyValue() reads back byte-exact, and cssmin and css-html-js-minify collapse whitespace inside strings, so their output can change the cascade where turbohtml’s round-trips. That rewrite is also the only reason rcssmin and cssmin end 0.2% to 0.3% ahead on bulma.css, whose declarations are almost entirely custom properties.

lightningcss, the Rust binding, is a cascade-aware optimizer: it drops declarations overridden elsewhere in the sheet and rewrites syntax for a browser-target set, so it reaches a smaller size than turbohtml on most of the corpus (turbohtml comes out ahead on normalize.css). That target-dependent optimization is the same idea as turbohtml’s baseline option carried further, and it is in scope. Its Rust engine runs 1.3 to 2.4 times slower than turbohtml across the corpus, its per-target cascade pass the added cost, and it rejects foundation.css with a parse error on a media query the WHATWG recovery rules accept, where turbohtml minifies all six. turbohtml gives the smallest value-safe output at the most compatible baseline and recovers from malformed input.

JavaScript minification

turbohtml.clean.minify_js() against the PyPI JavaScript minifiers it replaces – rjsmin (a regex substitution), jsmin (Crockford’s character state machine), css-html-js-minify (another regex pass), and calmjs.parse (a full ES5 parser with an obfuscating printer) – and the industry’s native minifiers as the size bar: terser (the JavaScript ecosystem’s reference), esbuild, and tdewolff/minify, each invoked through its command line. The inputs are real un-minified libraries, a size ladder every tool parses. turbohtml renames every local binding (function and class declarations included) and runs the structural folds, so it beats calmjs.parse’s heavier global obfuscation on size everywhere while running fifty to a hundred times faster, and its output lands within one percent of terser, esbuild, and tdewolff – the best minifiers available. It runs in-process, where each native tool pays its runtime’s process startup per file, so it finishes ahead end to end; only rjsmin is faster, and it, jsmin, and css-html-js-minify all do so by leaving output half again to twice the size. Each cell pairs a minifier’s output size with the time to produce it; both ratios are against turbohtml.

minify a JS library

turbohtml

tdewolff

esbuild

terser

rjsmin1

calmjs.parse

css-html-js-minify2

jsmin3

size

time

size

time

size

time

size

time

size

time

size

time

size

time

size

time

underscore 1.13 (67 kB)

19.3 kB

2.72 ms

19.3 kB (1.000x)

8.5 ms (3.2x ±7%)

19.3 kB (1.002x)

10.6 ms (4.0x ±6%)

19.2 kB (0.996x)

206 ms (75.8x ±5%)

34 kB (1.76x)

104 µs (0.1x ±6%)

20.9 kB (1.08x)

154 ms (56.5x ±5%)

34 kB (1.76x)

5.84 ms (2.2x ±3%)

34 kB (1.76x)

7.65 ms (2.9x ±3%)

backbone 1.6 (79 kB)

24.8 kB

1.6 ms

24.6 kB (0.99x)

8.61 ms (5.4x ±6%)

24.7 kB (0.998x)

10.4 ms (6.5x ±10%)

24.6 kB (0.99x)

181 ms (113x ±11%)

35.2 kB (1.42x)

89.2 µs (0.1x ±4%)

25.9 kB (1.05x)

179 ms (112x ±18%)

35.2 kB (1.42x)

6.26 ms (4.0x ±6%)

35.2 kB (1.42x)

8.27 ms (5.2x ±4%)

jquery 3.7 (279 kB)

87.8 kB

13.6 ms

87.4 kB (0.996x)

14.9 ms (1.1x ±5%)

87.7 kB (0.999x)

18.8 ms (1.4x ±4%)

87 kB (0.99x)

450 ms (33.1x ±4%)

141 kB (1.61x)

464 µs (0.1x ±3%)

93.2 kB (1.06x)

726 ms (53.4x ±10%)

148 kB (1.69x)

25.3 ms (1.9x ±3%)

141 kB (1.61x)

32.1 ms (2.4x ±2%)

lodash 4.17 (531 kB)

71.9 kB

12.8 ms

71.9 kB (1.000x)

17.6 ms (1.4x ±3%)

72.8 kB (1.01x)

22.4 ms (1.8x ±5%)

71.1 kB (0.99x)

485 ms (38.0x ±7%)

149 kB (2.07x)

761 µs (0.1x ±3%)

77 kB (1.07x)

679 ms (53.0x ±7%)

140 kB (1.95x)

43.7 ms (3.5x ±3%)

149 kB (2.07x)

44.5 ms (3.5x ±3%)

1 rjsmin strips whitespace and comments without parsing the source, so it performs none of the renaming or structural compression the parsed minifiers do: on jQuery it emits 141.1 kB where turbohtml emits 87.8 kB

2 css-html-js-minify strips whitespace and comments without parsing the source, so it performs none of the renaming or structural compression the parsed minifiers do: on jQuery it emits 141.1 kB where turbohtml emits 87.8 kB

3 jsmin strips whitespace and comments without parsing the source, so it performs none of the renaming or structural compression the parsed minifiers do: on jQuery it emits 141.1 kB where turbohtml emits 87.8 kB

Encoding detection

turbohtml.detect.detect() against the encoding detectors it replaces: chardet (the pure-Python prober ensemble), charset-normalizer (decode-and-score, what requests uses), faust-cchardet (the maintained C binding of uchardet; the original cchardet stops compiling at Python 3.11), resiliparse’s detect_encoding, and BeautifulSoup’s UnicodeDammit, benchmarked with the chardet backend it only sniffs with. turbohtml resolves certain input – a byte-order mark, a <meta> declaration, valid UTF-8, pure ASCII – structurally before any scoring, which is where the tens-to-nearly-2000x rows on the ASCII and pre-declared pages come from, and its chardetng frequency scoring keeps declaration-less single-byte text 3.9x-5.4x ahead of chardet.

resiliparse’s native scan is quickest on the small and CJK inputs, ahead of turbohtml by 1.1 to 2.3 times where the structural checks find nothing to short-circuit on; faust-cchardet (uchardet) leads only the Shift_JIS row, 1.6 times. On everything else turbohtml’s structural resolution runs away from the full-table scanners: uchardet spends 188 microseconds on the 4 kB UTF-8 stream turbohtml settles in five, and 30 milliseconds on the 95 kB pre-declared page it settles in under one. CJK is the case both native detectors keep, since turbohtml decodes each candidate encoding to score it and a CJK stream leaves several standing.

detect a byte stream’s encoding

turbohtml

resiliparse

charset-normalizer

chardet

BeautifulSoup (html.parser)

faust-cchardet

ascii (1 kB)

1.14 µs

1.03 µs (1.0x ±1%)

49.3 µs (43.3x ±4%)

146 µs (128x ±14%)

167 µs (147x ±9%)

1.18 µs (1.1x ±11%)

utf-8 russian (4 kB)

5.4 µs

3.08 µs (0.6x ±1%)

347 µs (64.3x ±2%)

161 µs (29.9x ±4%)

191 µs (35.4x ±9%)

188 µs (35.0x ±19%)

windows-1251 russian (4 kB)

176 µs

351 µs (2.0x ±2%)

302 µs (1.8x ±2%)

686 µs (3.9x ±12%)

682 µs (3.9x ±10%)

1.07 ms (6.1x ±2%)

windows-1252 french (4 kB)

179 µs

383 µs (2.2x ±6%)

1.02 ms (5.7x ±6%)

966 µs (5.4x ±12%)

961 µs (5.4x ±15%)

1.37 ms (7.7x ±7%)

shift_jis japanese (4 kB)

87.3 µs

38 µs (0.5x ±23%)

348 µs (4.0x ±9%)

734 µs (8.5x ±42%)

1.1 ms (12.7x ±29%)

54.4 µs (0.7x ±12%)

utf-8 page (95 kB)

908 ns

69.9 µs (77.0x ±16%)

590 µs (651x ±10%)

1.33 ms (1467x ±10%)

1.79 ms (1969x ±17%)

29.7 ms (32724x ±12%)

Legacy decoding

The WHATWG decoders against the CPython codecs they replaced: cp932 for Shift_JIS, cp1252, gb18030 and iso2022_jp. None of those is the spec’s decoder – the tables differ, the error handling differs, and Handle character encodings explains why no rename could have reconciled them – so the table prices the replacement rather than claiming the codecs were a substitute. Each case wraps prose in tags, the shape of a real page, since a decoder that walks markup one byte at a time pays for it: ASCII runs are copied out whole, and only ISO-2022-JP, whose escapes can reinterpret an ASCII byte, has to step through them. The gb18030 astral row, dense with four-byte sequences, is the one case where the CPython codec’s table lookup edges ahead.

decode a legacy byte stream

turbohtml

stdlib1

shift_jis japanese (8 kB)

5.4 µs

10.4 µs (2.0x ±3%)

gb18030 astral (16 kB)

22.8 µs

17.1 µs (0.8x ±3%)

windows-1252 french (9 kB)

2.58 µs

2.8 µs (1.1x ±3%)

gb18030 japanese (8 kB)

6.14 µs

15.5 µs (2.6x ±5%)

iso-2022-jp japanese (8 kB)

8.97 µs

15.2 µs (1.7x ±6%)

1 stdlib decodes with the nearest CPython codec under errors=replace, which is not the WHATWG decoder of that label: the two disagree on both the mapping tables and where decoding resumes after an error