From pyquery

pyquery latest releasepyquery supported Pythonspyquery licensepyquery monthly downloadspyquery total downloadspyquery GitHub starspyquery last commit

pyquery puts a jQuery-style fluent, chainable wrapper over lxml/cssselect, so you select and mutate a document with method chains. A single PyQuery object holds a matched set of nodes; calling it with a CSS selector, chaining .filter, .find, .eq, or .closest, and reading or writing attributes, text, HTML, and classes all return either a new wrapper or a scalar. It is a common pick for scraping and templating code that wants the DOM-manipulation feel of jQuery without a browser.

turbohtml covers that ground with turbohtml.query.Query, a fully type-annotated fluent wrapper whose selector, attribute, and class primitives run in the C extension over a native tree, so the same chaining idiom ports across with almost no rename and a large speed margin.

turbohtml vs pyquery

Dimension

turbohtml

pyquery

Scope

WHATWG HTML parser plus a jQuery-style Query wrapper over a native tree

jQuery-style wrapper only; parsing and the tree come from lxml

Feature breadth

CSS select/filter/traverse, attr/text/html/class ops, node-level mutation and XPath 1.0

CSS and XPath on the wrapper itself, broad jQuery method set, a network-fetching constructor

Performance

Selector and attribute primitives in C over a native tree; several to hundreds of times faster on the shared surface

Python wrapper delegating to lxml/cssselect

Typing

Fully type annotated with shipped stubs

No bundled type hints

Dependencies

Self-contained C extension, no Python runtime deps

Depends on lxml and cssselect

Maintenance

Actively developed alongside the parser

Mature, lightly maintained wrapper

Feature overlap

The shared surface ports one-to-one from a matched set built by calling the wrapper with a selector:

What turbohtml adds

  • A full WHATWG HTML parser in the same package, so parsing and querying share one native tree instead of delegating to lxml.

  • C-resident selector and attribute primitives, giving the large speed margin measured below.

  • Shipped type stubs for the whole surface, including Query and the node API.

  • No third-party runtime dependencies: the tree, selectors, and mutation all live in one self-contained C extension.

What pyquery has that turbohtml does not

  • A network-fetching constructor. PyQuery(url=...) fetches over HTTP for you. turbohtml has no equivalent; fetch with httpx (or any client) and hand the bytes to turbohtml.parse().

  • XPath on the fluent wrapper. pyquery exposes lxml’s .xpath(...) directly on a matched set. turbohtml’s Query is CSS-only; drop to the node-level xpath() (XPath 1.0) or the find() grammar via Query.items.

  • ``.wrap_all`` over a non-contiguous set. pyquery wraps any matched set in one container. turbohtml’s node API covers the tree-clean shapes (see below) but has no counterpart for an arbitrary scattered set; append() the nodes into a new element and place it yourself.

Performance

operation

turbohtml

pyquery

parse to a tree — wpt tiny (0.6 kB)

1.53 µs

6.19 µs (4.1x ±3%)

parse to a tree — wpt small (4 kB)

12.2 µs

38.4 µs (3.2x ±4%)

parse to a tree — wpt medium (9.6 kB)

32.5 µs

101 µs (3.2x ±7%)

parse to a tree — wpt large (92 kB)

276 µs

884 µs (3.2x ±3%)

parse to a tree — wpt CJK (124 kB)

532 µs

2.02 ms (3.9x ±7%)

parse to a tree — whatwg spec (235 kB)

519 µs

1.68 ms (3.3x ±3%)

parse to a tree — common tags (13 kB)

78.5 µs

225 µs (2.9x ±3%)

find every anchor — daring fireball (10 kB)

372 ns

9.77 µs (26.3x ±1%)

find every anchor — ars technica (56 kB)

811 ns

20 µs (24.8x ±2%)

find every anchor — mozilla blog (95 kB)

1.12 µs

29.5 µs (26.4x ±2%)

find every anchor — whatwg spec (235 kB)

1.3 µs

60 µs (46.3x ±2%)

select div a[href] — daring fireball (10 kB)

595 ns

29.8 µs (50.1x ±2%)

select div a[href] — ars technica (56 kB)

1.43 µs

127 µs (88.5x ±1%)

select div a[href] — mozilla blog (95 kB)

2.01 µs

801 µs (400x ±1%)

select div a[href] — whatwg spec (235 kB)

1.76 µs

1.42 ms (804x ±6%)

match each anchor against div a[href] — daring fireball (10 kB)

1.85 µs

431 µs (234x ±3%)

match each anchor against div a[href] — ars technica (56 kB)

4.23 µs

1.02 ms (242x ±2%)

match each anchor against div a[href] — mozilla blog (95 kB)

5.83 µs

1.41 ms (243x ±1%)

match each anchor against div a[href] — whatwg spec (235 kB)

6.46 µs

1.66 ms (258x ±2%)

find by text content — daring fireball (10 kB)

24.4 µs

50.6 µs (2.1x ±2%)

find by text content — ars technica (56 kB)

173 µs

236 µs (1.4x ±1%)

find by text content — mozilla blog (95 kB)

278 µs

441 µs (1.6x ±1%)

find by text content — whatwg spec (235 kB)

556 µs

1.17 ms (2.1x ±1%)

collect visible text — daring fireball (10 kB)

2.63 µs

127 µs (48.4x ±1%)

collect visible text — ars technica (56 kB)

13.5 µs

448 µs (33.2x ±1%)

collect visible text — mozilla blog (95 kB)

21.9 µs

906 µs (41.5x ±1%)

collect visible text — whatwg spec (235 kB)

83.7 µs

4.8 ms (57.4x ±1%)

serialize a parsed tree — daring fireball (10 kB)

6.52 µs

17.2 µs (2.7x ±2%)

serialize a parsed tree — ars technica (56 kB)

34.8 µs

76.7 µs (2.3x ±2%)

serialize a parsed tree — mozilla blog (95 kB)

67.4 µs

155 µs (2.3x ±3%)

serialize a parsed tree — whatwg spec (235 kB)

174 µs

377 µs (2.2x ±2%)

tag every link rel=nofollow — daring fireball (10 kB)

4.74 µs

37.9 µs (8.1x ±8%)

tag every link rel=nofollow — ars technica (56 kB)

16.6 µs

122 µs (7.4x ±25%)

tag every link rel=nofollow — mozilla blog (95 kB)

25.4 µs

221 µs (8.8x ±16%)

tag every link rel=nofollow — whatwg spec (235 kB)

51.4 µs

388 µs (7.6x ±6%)

class add/remove on every link — daring fireball (10 kB)

2.62 µs

41.5 µs (15.9x ±2%)

class add/remove on every link — ars technica (56 kB)

9.67 µs

108 µs (11.2x ±2%)

class add/remove on every link — mozilla blog (95 kB)

10.3 µs

137 µs (13.5x ±2%)

class add/remove on every link — whatwg spec (235 kB)

10.1 µs

178 µs (17.7x ±2%)

drop tags with content (remove) — daring fireball (10 kB)

23.2 µs

115 µs (5.0x ±6%)

drop tags with content (remove) — ars technica (56 kB)

112 µs

453 µs (4.1x ±1%)

drop tags with content (remove) — mozilla blog (95 kB)

252 µs

886 µs (3.6x ±1%)

drop tags with content (remove) — whatwg spec (235 kB)

650 µs

2.2 ms (3.4x ±2%)

unwrap tags keep content (strip_tags) — daring fireball (10 kB)

24.4 µs

132 µs (5.5x ±2%)

unwrap tags keep content (strip_tags) — ars technica (56 kB)

120 µs

525 µs (4.4x ±2%)

unwrap tags keep content (strip_tags) — mozilla blog (95 kB)

261 µs

973 µs (3.8x ±4%)

unwrap tags keep content (strip_tags) — whatwg spec (235 kB)

669 µs

2.59 ms (3.9x ±3%)

replace body inner HTML — daring fireball (10 kB)

2.12 µs

17.5 µs (8.3x ±2%)

replace body inner HTML — ars technica (56 kB)

7.8 µs

57.4 µs (7.4x ±6%)

replace body inner HTML — mozilla blog (95 kB)

11.7 µs

123 µs (10.6x ±2%)

replace body inner HTML — whatwg spec (235 kB)

35.3 µs

363 µs (10.3x ±2%)

replace body text — daring fireball (10 kB)

1.13 µs

14.7 µs (13.0x ±2%)

replace body text — ars technica (56 kB)

6.4 µs

54.1 µs (8.5x ±1%)

replace body text — mozilla blog (95 kB)

10.5 µs

118 µs (11.3x ±1%)

replace body text — whatwg spec (235 kB)

33 µs

357 µs (10.9x ±1%)

walk every descendant — daring fireball (10 kB)

3.37 µs

75.7 µs (22.5x ±1%)

walk every descendant — ars technica (56 kB)

13.3 µs

278 µs (20.9x ±3%)

walk every descendant — mozilla blog (95 kB)

28.5 µs

616 µs (21.6x ±3%)

walk every descendant — whatwg spec (235 kB)

97.5 µs

2.13 ms (21.9x ±1%)

fluent jQuery-style chain — daring fireball (10 kB)

1.74 µs

82.6 µs (47.6x ±1%)

fluent jQuery-style chain — ars technica (56 kB)

5.34 µs

175 µs (32.9x ±1%)

fluent jQuery-style chain — mozilla blog (95 kB)

8.09 µs

238 µs (29.5x ±3%)

fluent jQuery-style chain — whatwg spec (235 kB)

17.3 µs

302 µs (17.5x ±3%)

extract every link — daring fireball (10 kB)

13.5 µs

314 µs (23.4x ±21%)

extract every link — ars technica (56 kB)

45.7 µs

759 µs (16.7x ±18%)

extract every link — mozilla blog (95 kB)

75.5 µs

994 µs (13.2x ±14%)

extract every link — whatwg spec (235 kB)

66.4 µs

950 µs (14.4x ±7%)

absolutize every link — daring fireball (10 kB)

39.4 µs

410 µs (10.5x ±6%)

absolutize every link — ars technica (56 kB)

91.8 µs

989 µs (10.8x ±3%)

absolutize every link — mozilla blog (95 kB)

157 µs

1.44 ms (9.3x ±3%)

absolutize every link — whatwg spec (235 kB)

224 µs

1.84 ms (8.3x ±5%)

rewrite every link — daring fireball (10 kB)

3 µs

378 µs (127x ±25%)

rewrite every link — ars technica (56 kB)

10.7 µs

751 µs (70.1x ±6%)

rewrite every link — mozilla blog (95 kB)

19.9 µs

1.04 ms (52.2x ±5%)

rewrite every link — whatwg spec (235 kB)

33.3 µs

1.2 ms (36.2x ±4%)

social-card extraction — head

1.83 µs

64.1 µs (35.2x ±3%)

social-card extraction — article 8 KiB

21.9 µs

133 µs (6.1x ±1%)

extract @href per match — daring fireball (10 kB)

3.57 µs

237 µs (66.6x ±5%)

extract @href per match — ars technica (56 kB)

9.06 µs

542 µs (59.9x ±5%)

extract @href per match — mozilla blog (95 kB)

12.5 µs

764 µs (61.3x ±4%)

extract @href per match — whatwg spec (235 kB)

14.4 µs

939 µs (65.2x ±12%)

extract text per match — daring fireball (10 kB)

3.44 µs

143 µs (41.6x ±12%)

extract text per match — ars technica (56 kB)

8.73 µs

339 µs (38.8x ±5%)

extract text per match — mozilla blog (95 kB)

14.6 µs

507 µs (34.7x ±15%)

extract text per match — whatwg spec (235 kB)

14.6 µs

598 µs (40.9x ±14%)

extract URL hints — base_url / get_base_url

1.17 µs

13 µs (11.2x ±3%)

extract URL hints — meta_refresh / get_meta_refresh

1.21 µs

20.9 µs (17.3x ±3%)

extract filtered page links — daring fireball (10 kB)

144 µs

519 µs (3.7x ±16%)

extract filtered page links — ars technica (56 kB)

363 µs

1.36 ms (3.8x ±10%)

extract filtered page links — mozilla blog (95 kB)

619 µs

2.15 ms (3.5x ±8%)

extract filtered page links — whatwg spec (235 kB)

1.06 ms

3.19 ms (3.0x ±13%)

The whole shared surface – chaining a select/filter/read, setting content, bulk-editing tags, and reading a value off every match – runs several to hundreds of times faster because the wrapper edits its native tree in C and skips a redundant de-duplication when a chain starts from a single node, where pyquery drives lxml under its jQuery-style wrapper.

How to migrate

Build a Query from a parsed document and call it with a selector; the method chains port almost name for name:

from turbohtml import parse
from turbohtml.query import Query

query = Query(parse("<div><a href='/u'>l</a><a>m</a></div>"))
print(query("a").filter("[href]").eq(0).add_class("seen").attr("href"))
print([anchor.text() for anchor in query("a").items()])
/u
['l', 'm']

pyquery

turbohtml

pq = PyQuery(html)

Query(parse(html))

pq("div.foo"), pq("a").find("span")

query("div.foo"), query("a").find("span")

.filter(sel), .eq(i), .closest(sel)

the same names

.attr("href"), .attr("k", "v")

the same names

.text(), .html()

the same names

.add_class(c), .remove_class(c), .toggle_class(c), .has_class(c)

add_class(), remove_class(), toggle_class(), has_class() (also on Query)

.parent(), .children(), .siblings()

the same names

iterating for item in pq("a").items()

for item in query("a").items()

jQuery pq("script").remove(), pq(".box b").remove()

node.remove("script"), node.remove(".box b")

jQuery $(".box b").contents().unwrap() (drop the tag, keep the text)

node.strip_tags(".box b")

pyquery’s .wrap_all(html) wraps a whole matched set in one new container in place; the node API has two methods for the shapes that fit a tree model cleanly. wrap_children() boxes every child of a container, and wrap_siblings() wraps a node and the contiguous run of siblings after it (through an until node, or to the last sibling), so query("p").wrap_all("<div/>") over a run of adjacent paragraphs becomes first.wrap_siblings(Element("div"), until=last):

pyquery

turbohtml

pq("section").contents().wrap_all("<div/>")

section.wrap_children(Element("div"))

pq(run).wrap_all("<div/>") over a contiguous run

first.wrap_siblings(Element("div"), until=last)

pyquery’s content setters – .html(markup) reparses a matched element’s children and .text(s) replaces them with one verbatim text node – map onto three element methods. set_inner_html() parses the markup as a fragment in the element’s context and replaces its children; set_text() replaces them with one verbatim text node; and insert_adjacent_html() splices a parsed fragment at a DOM position (the .append(markup) / insertAdjacentHTML shape):

pyquery

turbohtml

pq(el).html(markup)

el.set_inner_html(markup)

pq(el).text(s)

el.set_text(s)

pq(el).append(markup)

el.insert_adjacent_html("beforeend", markup)

Gotchas and pitfalls

  • .wrap_all over an arbitrary, non-contiguous set of nodes has no single node-method counterpart (the set has no shared anchor to place the wrapper at); wrap the contiguous run, or append() the scattered nodes into one new element and place it yourself.

  • pyquery’s network-fetching constructor (PyQuery(url=...)) is out of scope: fetch with httpx (or any client) and hand the bytes to turbohtml.parse().

  • pyquery exposes lxml’s .xpath(...) on the fluent wrapper itself; turbohtml’s Query is CSS-only, so an XPath chain drops to the node-level xpath() (XPath 1.0) or the find() grammar via Query.items.

  • turbohtml parses to the WHATWG spec, so a malformed document is fixed up exactly as a browser would (implied <tbody>, reparented <head> content); pyquery’s tree follows lxml’s HTML parser, which can differ on the same broken input.