############# Performance ############# Every number here comes from `pyperf `_ on CPython 3.14.6 (a release build) on an Apple M4 running macOS 26. pyperf runs each case in isolated worker processes and reports the mean; the harness prints the run-to-run standard deviation beside it as ``±N%`` so a real gap reads apart from noise, and these tables quote the mean. For the lowest-noise figures, tune the machine with ``pyperf system tune`` first (and ``sudo pyperf system reset`` after); pyperf options like ``--rigorous`` or ``--affinity`` pass straight through after the ``tox -e bench`` command. Operations that mutate the tree they are handed -- the edits, the content setters, link absolutization -- are timed on a fresh parse rebuilt before each iteration (the rebuild itself untimed), so the figure is the repeatable cost of the mutation alone rather than a tree a prior iteration already changed; read-path operations reuse one cached parse and time only the query. The corpora are real documents: `Project Gutenberg's War and Peace `_, the `WHATWG HTML specification source `_, the `ECMAScript specification `_, a size-weighted sample of `web-platform-tests `_ pages for the parse and tokenize suites, and, for the read-path suites, real saved web pages -- a blog, a news article, and a product blog from the `mozilla/readability `_ test corpus -- so the selector, link, and edit operations run against genuine nested structure rather than the layout fixtures, which carry none. The harness benchmarks each competitor in its own isolated ``uv`` venv -- turbohtml in a venv of its own as the shared baseline -- so one library's dependency pins never perturb another's. Every table below is one harness operation, so each is reproducible with ``tox -e bench ``, where the command is ``core`` (turbohtml's own baseline for every operation), an operation name (the cross-competitor table), a package name (that competitor's own report), or ``all``. By default the turbohtml baseline is a plain wheel, which builds quickly for iterating; pass ``--pgo`` before the command (``tox -e bench -- --pgo core``) to build it instead with the shipped profile-guided, link-time-optimized release recipe, so the baseline reads as what a release ships rather than a plain build, at the cost of a much slower build. Most operations are a single call; a few aggregate workloads (``build``, ``build-e``) sweep a size, and the ``construct`` and ``emit`` breakdowns decompose that write path into the constructor and the serializer in isolation. Numbers vary with input and hardware. ********** Escaping ********** :func:`turbohtml.escape` against the standard library's :func:`python:html.escape`. It gains the most on text that needs little escaping, where the SIMD scan classifies sixteen bytes at a time and copies clean stretches in bulk; the gap narrows on tiny strings, where call overhead dominates. .. bench-table:: :file: bench/escaping.json ******************* Markup (escaping) ******************* :func:`turbohtml.migration.markupsafe.escape` against `markupsafe `_'s own C escape, both returning a ``Markup``. The inputs are the small, mostly-clean strings a template engine interpolates under autoescape, markupsafe's hottest path. turbohtml builds the safe string in C in a single call, where markupsafe pays a Python ``escape`` frame and ``Markup`` construction per call, so it runs roughly two to four times faster. .. bench-table:: :file: bench/markup-escaping.json The other ``Markup`` operations race markupsafe's own ``Markup`` of the same method. ``striptags`` and ``unescape`` run on turbohtml's tokenizer and HTML5 reference resolution where markupsafe scans with a regex, and ``format`` and ``join`` escape each untrusted operand through the same C ``escape``. .. bench-table:: :file: bench/markup-escaping-2.json ********* Linkify ********* :func:`turbohtml.clean.linkify` against `bleach `_'s ``linkify``, the HTML-aware linkifier it succeeds, and `linkify-it-py `_, the pure-Python scanner markdown-it-py pulls in. bleach and turbohtml both parse the HTML and rewrite it; linkify-it-py only finds the matches and does not rewrite, so it does strictly less work, yet turbohtml is faster than both. The C candidate scan and turbohtml's own tree carry it past bleach's html5lib pass by five to eighteen times. .. bench-table:: :file: bench/linkify.json The detection primitive on its own, :meth:`turbohtml.clean.LinkDetector.find` against ``LinkifyIt().match`` and :meth:`~turbohtml.clean.LinkDetector.has_link` against ``LinkifyIt().test``, scans a run of plain text and returns the spans or a boolean without rewriting any HTML, so this isolates the C scan from the full linkify rewrite above. The ``has_link`` prose row is close because ``test`` short-circuits on the first link near the start. .. bench-table:: :file: bench/linkify-2.json ********** Sanitize ********** :func:`turbohtml.clean.sanitize` against four sanitizers. Three share its allowlist model, where only listed tags and attributes survive, so a vector nobody anticipated is dropped by default: `nh3 `_ (the Rust ammonia binding), `bleach `_ (its end-of-life predecessor, on html5lib), and `html-sanitizer `_ (an allowlist over lxml). The fourth, `lxml-html-clean `_ (the externalized ``lxml.html.clean.Cleaner``), is a blocklist: it strips the constructs it knows are dangerous and lets the rest through, a model lxml itself flagged as hard to keep safe. The inputs are realistic user content with a few disallowed tags and a dangerous attribute mixed in. turbohtml runs the whole filtering walk in C and leads every alternative, but the model matters more than the microseconds. Prefer an allowlist, since a blocklist passes anything it did not think to name. .. bench-table:: :file: bench/sanitize.json ********** Markdown ********** :meth:`turbohtml.Node.to_markdown` against `markdownify `_ (on BeautifulSoup) and `html2text `_ (a streaming ``HTMLParser`` subclass). All three take an HTML string and return Markdown, so each parses first; turbohtml parses to the WHATWG tree and walks it in C, where the others build and convert in Python. The single C pass converts a page in tens of microseconds, two orders of magnitude ahead of markdownify. The ``configured`` row turns the option surface on in all three (underscore emphasis, reference links, padded tables, full escaping), and turbohtml stays ahead by the same margin. .. bench-table:: :file: bench/markdown.json The ``google_doc`` row reads the inline-CSS styling a Google Docs export carries (html2text's google_doc mode); markdownify has no equivalent. ***************** Structured data ***************** :meth:`turbohtml.Document.structured_data` against `extruct `_, the scraper toolkit it succeeds, extracting JSON-LD, Microdata, and OpenGraph from a product page that carries all three. Both start from the raw HTML string, so each parses first; extruct builds an lxml tree and runs a separate extractor per syntax, where turbohtml parses to the WHATWG tree and gathers every format in one C walk, handing back the typed :class:`~turbohtml.StructuredData` record. The single pass runs roughly nine to eleven times faster. .. bench-table:: :file: bench/structured-data.json ******** Tables ******** :meth:`turbohtml.Node.tables` and :meth:`turbohtml.Element.records` against `pandas `_'s ``read_html``, the one-call table reader scrapers reach for. Both parse the HTML and extract every ````, resolving ``rowspan`` and ``colspan`` into a rectangular grid; ``read_html`` returns a ``DataFrame`` per table and pulls in NumPy, where turbohtml runs the cell-grid walk in C and hands back plain ``list`` and ``dict`` objects with no added dependency. The ``rows`` row times :meth:`~turbohtml.Node.tables` (every table as ``list[list[str]]``) and the ``records`` row times :meth:`~turbohtml.Element.records` (the first table keyed by its header), each over a four-column table of the given height. The single C pass leads from roughly twenty times on the largest table to over a hundred and thirty on the smallest, where pandas pays its fixed per-frame construction cost. .. bench-table:: :file: bench/tables.json ******************** Article extraction ******************** :meth:`turbohtml.Node.article` against `trafilatura `_, `readability-lxml `_, `newspaper3k `_, `goose3 `_, `readabilipy `_, and `news-please `_, the article extractors it succeeds. Each scores the dominant content body and (trafilatura, newspaper3k, goose3, and news-please) harvests the page metadata beside it; the lxml-backed four build their tree in Python first, readabilipy's Python mode parses with html5lib into BeautifulSoup and cleans without scoring, news-please merges the votes of several such extractors, and turbohtml does the scoring and the harvest in one C pass over the parsed tree. The inputs are full pages -- navigation, a scored article, and a footer -- so the boilerplate the heuristic discounts is part of the measured cost. .. bench-table:: :file: bench/article-extraction.json **************************** Boilerplate classification **************************** :func:`turbohtml.extract.boilerplate` against `justext `_ and `boilerpy3 `_, the per-block boilerplate classifiers. All three segment the page into units and mark each good or boilerplate; justext scores every paragraph in Python over an lxml tree (length, link density, stopword density), boilerpy3 classifies the blocks of its own SAX stream with boilerpipe's rules, and turbohtml scores the tree once in C and classifies the units in a thin Python layer. The inputs are the article-extraction pages, so the navigation and footer each classifier must reject are part of the measured cost. .. bench-table:: :file: bench/boilerplate-classification.json ***************** Date extraction ***************** :func:`turbohtml.extract.dates` against `htmldate `_, the standalone publication-date finder. Both read the same signals -- publication/modification ```` tags, JSON-LD, ```` context, where the WHATWG algorithm's table rules apply. turbohtml runs the same C engine it uses for whole documents, so it parses the fragment three times faster than lxml and roughly seventy-five times faster than the pure-Python html5lib. .. bench-table:: :file: bench/fragment-parsing.json ********** Querying ********** Each library parses the document once, then the timed call runs one query. ``find`` collects every ```` element the way each library reaches for it (turbohtml's :meth:`~turbohtml.Node.find_all`, lxml's XPath ``findall``, selectolax's and BeautifulSoup's selectors). A tag-only query resolves the name to an interned atom and walks the subtree comparing integers, with no per-element string built and no matcher dispatch, so it runs ahead of lxml's C XPath engine and many times ahead of selectolax, parsel (Scrapy's cssselect-over-libxml2 selector library), and BeautifulSoup. .. bench-table:: :file: bench/querying.json ``select`` runs the CSS selector ``div a[href]`` (turbohtml's :meth:`~turbohtml.Node.select`, lxml's `cssselect `_, selectolax's ``css``, parsel's ``css``, BeautifulSoup's `soupsieve `_). Because turbohtml compiles the selector against the tree once and then matches by comparing interned integer atoms, it stays in the low microseconds across these pages. lxml and parsel re-translate the selector to XPath through cssselect on every call, which scales with the document and trails by tens of times on the small blog up to roughly seven hundred fifty times on the spec; BeautifulSoup's soupsieve is hundreds to more than fifteen hundred times behind, while selectolax, the other compiled engine, stays closest at roughly twelve to forty-five times. .. bench-table:: :file: bench/querying-2.json The relational ``:has()`` pseudo-class is the costliest selector to evaluate, since a naive matcher rescans each candidate's subtree. turbohtml runs ``div:has(a)`` against the same pages and leads every alternative: tens of times faster than lxml and selectolax on the smaller pages, narrowing to single digits on the link-dense mozilla blog where the relational match itself does real work, while BeautifulSoup trails by hundreds of times throughout. The matcher walks each anchor's descendants once and skips the sibling scan for descendant and child relationships, so the relational lookup keeps the same interned-atom comparison the flat selectors use. .. bench-table:: :file: bench/querying-3.json Per-element matching runs each anchor on the page through a compiled ``div a[href]`` matcher -- the shape a soupsieve port hits through :mod:`turbohtml.query` and its :meth:`Matcher.match ` (soupsieve is the only competitor with a compiled per-element match). turbohtml answers each test with the same interned-atom comparison its ``select`` uses, walking the ancestor chain once per candidate, where soupsieve re-interprets the parsed selector in Python per element, so the sweep runs 78 to 148 times faster across these pages. .. bench-table:: :file: bench/matching.json A text-content search runs through :meth:`~turbohtml.Node.find_all` with ``text=`` (a regex matched against each element's collected subtree text), raced against ``BeautifulSoup.find_all(string=...)``; lxml, selectolax, and parsel expose no equivalent, so this is a two-way race. When the ``text=`` filter is a plain string or a literal (no regex metacharacters, case-sensitive) compiled pattern, turbohtml gathers each candidate's collected text and matches it in C -- no Python ``str`` built, no per-element ``re.search`` call -- where BeautifulSoup's ``find_all(string=...)`` walks the tree in Python, so turbohtml now leads on every page (it previously trailed on the larger ones). A case-insensitive or otherwise non-literal pattern keeps the per-element Python path. .. bench-table:: :file: bench/querying-4.json XPath 1.0 evaluation runs through :meth:`~turbohtml.Node.xpath`, raced against lxml's libxml2 engine, the XPath that parsel, pyquery, and html5-parser all wrap (selectolax and BeautifulSoup have none). One expression per feature class (name tests, the ``//`` abbreviation, attribute, positional, and arithmetic predicates, string and aggregate functions, a reverse axis, a union, and a computed name test) runs over the 9.6 kB wpt page below; ``tox -e bench xpath`` repeats the sweep across every page size. turbohtml compiles each expression against the tree once, resolves name tests to interned atoms, and folds ``//`` to a single ``descendant`` walk, so it leads across the surface. The exception is a predicate that references ``position()`` (``[1]`` or ``position() <= 3``): it pins the result to proximity order and disables the ``//`` collapse, so on the largest pages lxml's streaming evaluation closes the gap. The last eight rows are the lxml/parsel options the parity work added: a ``$variable`` binding, an EXSLT ``re:test`` predicate (turbohtml's Python :mod:`re` against lxml's C libexslt), an EXSLT ``set:distinct`` node-set reduction (built-in C dispatch on both sides, so it races C against C), a ``smart_strings`` attribute read, a custom ``extensions=`` function, an ``extensions=`` function whose return becomes a node-set feeding a later ``/@href`` step, a ``namespaces=`` prefix binding that resolves ``//svg:rect`` against ``{"svg": ".../2000/svg"}`` over a page carrying an SVG block, and a node-set ``$variable`` bound from a prior result (``$rows/div``, with ``rows`` reused from an earlier ``//div`` query) fed into a later path step. turbohtml still leads, since lxml resolves the namespace map and option set on every call. The last row precompiles the expression once with :class:`~turbohtml.XPath` and re-evaluates it, lxml's ``etree.XPath`` doing the same: both skip the per-call parse :meth:`~turbohtml.Node.xpath` pays, and turbohtml's compiled program stays ahead per evaluation. .. bench-table:: :file: bench/querying-5.json ************ Node paths ************ :meth:`turbohtml.Element.css_path` and :meth:`~turbohtml.Element.xpath_path` return the unique locator that re-finds an element from the document root -- a CSS selector and a positional XPath -- against lxml's ``getroottree().getpath()``, the libxml2 path builder devtools' "copy selector" mirrors. lxml emits only the positional XPath, so ``getpath`` pairs with both turbohtml methods. Each timed call walks every element in a pre-parsed page and serializes its path. Both methods lead ``getpath`` by roughly five to six times across these pages, narrowing to under threefold on the spec. :meth:`~turbohtml.Element.css_path` previously rescanned the whole document to test each element's id uniqueness, an O(N\ :sup:`2`) cost over a page that made it slower than ``getpath`` on id-heavy pages; a cached per-tree id-occurrence map (dropped with the element index on any mutation) now answers that test in O(1), so ``css_path`` keeps pace with the positional ``xpath_path``. The ratio on ``getpath`` is against ``xpath_path``, the like-for-like locator. .. bench-table:: :file: bench/node-paths.json ************** Text content ************** The ``text`` suite collects the visible text two ways. First, the raw text join off a pre-parsed tree, the ``get_text`` pass: turbohtml's :attr:`~turbohtml.Node.text` property concatenates every descendant text run, against lxml's ``text_content()``, selectolax's ``text()``, and BeautifulSoup's ``get_text()``. turbohtml gathers the runs in one C walk into a buffer reserved up front, so it leads lxml by a small margin and selectolax and BeautifulSoup by roughly an order of magnitude. parsel exposes no node-level text collector, so it sits out. .. bench-table:: :file: bench/text-content.json Second, the layout-aware string-to-text extraction: :meth:`turbohtml.Node.to_text` against `inscriptis `_, the layout-aware HTML-to-text renderer it succeeds, `html-text `_, Zyte's plainer visible-text extractor, and `resiliparse `_'s ``extract_plain_text``. inscriptis and html-text both build an lxml tree in Python and resiliparse renders text off the lexbor tree it parses to, where turbohtml does the whole layout in one C walk; inscriptis additionally lays tables out as aligned columns, which html-text and resiliparse skip. .. bench-table:: :file: bench/text-content-2.json The ``collapsed`` row turns layout guessing off: turbohtml joins the :attr:`~turbohtml.Node.stripped_strings` word stream against html-text's ``extract_text(guess_layout=False)``; inscriptis and resiliparse have no comparable collapsed mode. The ``main`` row strips page boilerplate first, :meth:`~turbohtml.Node.main_text` against resiliparse's ``extract_plain_text(main_content=True)``. The ``annotated`` row labels matching elements with spans through :meth:`~turbohtml.Node.to_annotated_text` against inscriptis's ``get_annotated_text``; html-text and resiliparse have no annotation surface, so they sit out that row. ***************** Tree navigation ***************** Walking every descendant of a parsed tree: turbohtml's :attr:`~turbohtml.Node.descendants` iterator against lxml's ``iterdescendants()`` and BeautifulSoup's ``descendants``. The ``list(el)``, ``iterdescendants()``, and ``iterancestors()`` family ports to :attr:`~turbohtml.Node.children`, :attr:`~turbohtml.Node.descendants`, and :attr:`~turbohtml.Node.ancestors`; the descendant walk is the dominant case. Each timed call consumes the whole iterator, where turbohtml yields interned nodes straight from the arena faster than lxml's libxml2 proxy objects and BeautifulSoup's Python ``NavigableString`` chain. The descendant walk is one of BeautifulSoup's leaner paths, so the margin is narrower here than on the query and serialize suites. selectolax exposes no document-wide descendant iterator, so it has no entry. .. bench-table:: :file: bench/tree-navigation.json ************* Serializing ************* Serializing a parsed document back to HTML: turbohtml's :attr:`~turbohtml.Node.html`, lxml's ``tostring``, selectolax's ``html``, and BeautifulSoup's ``decode``. turbohtml scans each text run for the next character that needs escaping (two code points at a time with the same SWAR lane probes :func:`~turbohtml.escape` uses) and bulk-copies the clean spans, recovering each special's position from the lane mask, and reserves the whole-document buffer up front so the output grows in one allocation. It serializes four to six times faster than lxml, about four times faster than selectolax, and forty-five to sixty times faster than BeautifulSoup. .. bench-table:: :file: bench/serializing.json *********** Minifying *********** Minifying a document with :func:`turbohtml.clean.minify`: parse, then serialize once with every fold engaged (collapsing insignificant whitespace, omitting the WHATWG-optional tags, unquoting attributes, and stripping comments), against `minify-html `_'s Rust minifier on the same folds (its CSS and JS minification left off for a like-for-like comparison), the pure-Python `htmlmin `_ and ``css-html-js-minify``, and the native CLI minifiers `html-minifier-terser `_ and `tdewolff/minify `_. turbohtml parses and emits in C through one preallocated buffer, so with the parse included it runs roughly two to three times faster than minify-html and sixteen to seventy times faster than the pure-Python pair. html-minifier-terser and tdewolff are invoked through their command line, so their millisecond timings are dominated by process startup; the clean comparison against them is output size. turbohtml lands within about two percent of html-minifier-terser on the structural folds both apply; minify-html folds more aggressively for roughly three to ten percent smaller, and tdewolff goes further still by also minifying the inline CSS and JavaScript turbohtml leaves untouched here. .. bench-table:: :file: bench/minifying.json ********** Building ********** The write path: construct a ``