From pyquery

pyquery latest releasepyquery supported Pythonspyquery licensepyquery monthly downloadspyquery total downloadspyquery GitHub starspyquery last commit

pyquery puts a jQuery-style fluent, chainable wrapper over lxml/cssselect, so you select and mutate a document with method chains. A single PyQuery object holds a matched set of nodes; calling it with a CSS selector, chaining .filter, .find, .eq, or .closest, and reading or writing attributes, text, HTML, and classes all return either a new wrapper or a scalar. It is a common pick for scraping and templating code that wants the DOM-manipulation feel of jQuery without a browser.

turbohtml covers that ground with turbohtml.query.Query, a fully type-annotated fluent wrapper whose selector, attribute, and class primitives run in the C extension over a native tree, so the same chaining idiom ports across with almost no rename and a large speed margin.

turbohtml vs pyquery

Dimension

turbohtml

pyquery

Scope

WHATWG HTML parser plus a jQuery-style Query wrapper over a native tree

jQuery-style wrapper only; parsing and the tree come from lxml

Feature breadth

CSS select/filter/traverse, attr/text/html/class ops, node-level mutation and XPath 1.0

CSS and XPath on the wrapper itself, broad jQuery method set, a network-fetching constructor

Performance

Selector and attribute primitives in C over a native tree; several to ~100x faster on the shared surface

Python wrapper delegating to lxml/cssselect

Typing

Fully type annotated with shipped stubs

No bundled type hints

Dependencies

Self-contained C extension, no Python runtime deps

Depends on lxml and cssselect

Maintenance

Actively developed alongside the parser

Mature, lightly maintained wrapper

Feature overlap

The shared surface ports one-to-one from a matched set built by calling the wrapper with a selector:

What turbohtml adds

  • A full WHATWG HTML parser in the same package, so parsing and querying share one native tree instead of delegating to lxml.

  • C-resident selector and attribute primitives, giving the large speed margin measured below.

  • Shipped type stubs for the whole surface, including Query and the node API.

  • No third-party runtime dependencies: the tree, selectors, and mutation all live in one self-contained C extension.

What pyquery has that turbohtml does not

  • A network-fetching constructor. PyQuery(url=...) fetches over HTTP for you. turbohtml has no equivalent; fetch with httpx (or any client) and hand the bytes to turbohtml.parse().

  • XPath on the fluent wrapper. pyquery exposes lxml’s .xpath(...) directly on a matched set. turbohtml’s Query is CSS-only; drop to the node-level xpath() (XPath 1.0) or the find() grammar via Query.items.

  • ``.wrap_all`` over a non-contiguous set. pyquery wraps any matched set in one container. turbohtml’s node API covers the tree-clean shapes (see below) but has no counterpart for an arbitrary scattered set; append() the nodes into a new element and place it yourself.

Performance

operation

turbohtml

pyquery

parse to a tree — wpt tiny (0.6 kB)

1.21 µs

7.18 µs (6.0x)

parse to a tree — wpt small (4 kB)

9.57 µs

31.5 µs (3.3x)

parse to a tree — wpt medium (9.6 kB)

24.2 µs

81.4 µs (3.4x)

parse to a tree — wpt large (92 kB)

207 µs

874 µs (4.3x)

parse to a tree — wpt CJK (124 kB)

407 µs

1.43 ms (3.6x)

parse to a tree — whatwg spec (235 kB)

405 µs

1.69 ms (4.2x)

find every anchor — daring fireball (10 kB)

375 ns

9.97 µs (26.6x)

find every anchor — ars technica (56 kB)

828 ns

10.1 µs (12.3x)

find every anchor — mozilla blog (95 kB)

1.18 µs

31.8 µs (27.0x)

find every anchor — whatwg spec (235 kB)

1.32 µs

61.9 µs (47.1x)

select div a[href] — daring fireball (10 kB)

639 ns

29.5 µs (46.2x)

select div a[href] — ars technica (56 kB)

1.46 µs

18.5 µs (12.7x)

select div a[href] — mozilla blog (95 kB)

2.07 µs

807 µs (391x)

select div a[href] — whatwg spec (235 kB)

1.74 µs

1.37 ms (786x)

find by text content — daring fireball (10 kB)

42.9 µs

51.3 µs (1.2x)

find by text content — ars technica (56 kB)

431 µs

256 µs (0.6x)

find by text content — mozilla blog (95 kB)

329 µs

443 µs (1.4x)

find by text content — whatwg spec (235 kB)

638 µs

1.4 ms (2.2x)

collect visible text — daring fireball (10 kB)

2.69 µs

128 µs (47.7x)

collect visible text — ars technica (56 kB)

13.4 µs

11.1 µs (0.9x)

collect visible text — mozilla blog (95 kB)

23.3 µs

926 µs (39.8x)

collect visible text — whatwg spec (235 kB)

76.8 µs

4.91 ms (63.9x)

serialize a parsed tree — daring fireball (10 kB)

7.14 µs

17.1 µs (2.4x)

serialize a parsed tree — ars technica (56 kB)

40.7 µs

76.2 µs (1.9x)

serialize a parsed tree — mozilla blog (95 kB)

79.2 µs

156 µs (2.0x)

serialize a parsed tree — whatwg spec (235 kB)

209 µs

382 µs (1.9x)

tag every link rel=nofollow — daring fireball (10 kB)

3.32 µs

20.8 µs (6.3x)

tag every link rel=nofollow — ars technica (56 kB)

13.2 µs

63.5 µs (4.9x)

tag every link rel=nofollow — mozilla blog (95 kB)

21 µs

61.8 µs (3.0x)

tag every link rel=nofollow — whatwg spec (235 kB)

42.5 µs

333 µs (7.9x)

class add/remove on every link — daring fireball (10 kB)

2.2 µs

42.9 µs (19.6x)

class add/remove on every link — ars technica (56 kB)

8.55 µs

10.2 µs (1.2x)

class add/remove on every link — mozilla blog (95 kB)

8.48 µs

146 µs (17.2x)

class add/remove on every link — whatwg spec (235 kB)

8.08 µs

184 µs (22.8x)

drop tags with content (remove) — daring fireball (10 kB)

25.3 µs

166 µs (6.6x)

drop tags with content (remove) — ars technica (56 kB)

125 µs

349 µs (2.8x)

drop tags with content (remove) — mozilla blog (95 kB)

283 µs

1.24 ms (4.4x)

drop tags with content (remove) — whatwg spec (235 kB)

687 µs

2.97 ms (4.4x)

unwrap tags keep content (strip_tags) — daring fireball (10 kB)

25.8 µs

184 µs (7.2x)

unwrap tags keep content (strip_tags) — ars technica (56 kB)

127 µs

335 µs (2.7x)

unwrap tags keep content (strip_tags) — mozilla blog (95 kB)

286 µs

1.22 ms (4.3x)

unwrap tags keep content (strip_tags) — whatwg spec (235 kB)

742 µs

3.04 ms (4.2x)

replace body inner HTML — daring fireball (10 kB)

2.33 µs

18 µs (7.8x)

replace body inner HTML — ars technica (56 kB)

9.43 µs

13.5 µs (1.5x)

replace body inner HTML — mozilla blog (95 kB)

15.6 µs

121 µs (7.8x)

replace body inner HTML — whatwg spec (235 kB)

45.4 µs

369 µs (8.2x)

replace body text — daring fireball (10 kB)

1.19 µs

32.8 µs (27.7x)

replace body text — ars technica (56 kB)

7.72 µs

12.9 µs (1.7x)

replace body text — mozilla blog (95 kB)

13 µs

124 µs (9.6x)

replace body text — whatwg spec (235 kB)

39.8 µs

371 µs (9.4x)

walk every descendant — daring fireball (10 kB)

3.34 µs

76.6 µs (23.0x)

walk every descendant — ars technica (56 kB)

13.8 µs

243 µs (17.7x)

walk every descendant — mozilla blog (95 kB)

28.2 µs

609 µs (21.6x)

walk every descendant — whatwg spec (235 kB)

101 µs

2.14 ms (21.3x)

fluent jQuery-style chain — daring fireball (10 kB)

3.68 µs

83.8 µs (22.8x)

fluent jQuery-style chain — ars technica (56 kB)

11.2 µs

22.5 µs (2.1x)

fluent jQuery-style chain — mozilla blog (95 kB)

16.6 µs

242 µs (14.6x)

fluent jQuery-style chain — whatwg spec (235 kB)

28.9 µs

304 µs (10.5x)

extract every link — daring fireball (10 kB)

8.66 µs

165 µs (19.1x)

extract every link — ars technica (56 kB)

27.7 µs

10.4 µs (0.4x)

extract every link — mozilla blog (95 kB)

51.5 µs

528 µs (10.3x)

extract every link — whatwg spec (235 kB)

84.8 µs

674 µs (8.0x)

absolutize every link — daring fireball (10 kB)

82 µs

453 µs (5.6x)

absolutize every link — ars technica (56 kB)

220 µs

16.6 µs (0.1x)

absolutize every link — mozilla blog (95 kB)

428 µs

1.51 ms (3.6x)

absolutize every link — whatwg spec (235 kB)

282 µs

1.9 ms (6.8x)

rewrite every link — daring fireball (10 kB)

3.24 µs

326 µs (101x)

rewrite every link — ars technica (56 kB)

13.4 µs

10.5 µs (0.8x)

rewrite every link — mozilla blog (95 kB)

23.1 µs

1.12 ms (48.5x)

rewrite every link — whatwg spec (235 kB)

41.9 µs

1.27 ms (30.3x)

social-card extraction — head

1.9 µs

76.7 µs (40.3x)

social-card extraction — article 8 KiB

25.3 µs

169 µs (6.8x)

extract @href per match — daring fireball (10 kB)

2.58 µs

213 µs (82.4x)

extract @href per match — ars technica (56 kB)

6.65 µs

10.8 µs (1.7x)

extract @href per match — mozilla blog (95 kB)

9.23 µs

548 µs (59.4x)

extract @href per match — whatwg spec (235 kB)

9.81 µs

664 µs (67.7x)

extract text per match — daring fireball (10 kB)

2.69 µs

97.4 µs (36.2x)

extract text per match — ars technica (56 kB)

6.68 µs

10.3 µs (1.6x)

extract text per match — mozilla blog (95 kB)

10.2 µs

351 µs (34.4x)

extract text per match — whatwg spec (235 kB)

10.9 µs

380 µs (34.9x)

extract URL hints — base_url / get_base_url

1.25 µs

18.4 µs (14.8x)

extract URL hints — meta_refresh / get_meta_refresh

1.31 µs

26.3 µs (20.2x)

extract filtered page links — daring fireball (10 kB)

117 µs

164 µs (1.5x)

extract filtered page links — ars technica (56 kB)

280 µs

10.6 µs (0.1x)

extract filtered page links — mozilla blog (95 kB)

471 µs

528 µs (1.2x)

extract filtered page links — whatwg spec (235 kB)

786 µs

653 µs (0.9x)

The whole shared surface – chaining a select/filter/read, setting content, bulk-editing tags, and reading a value off every match – runs several to a hundred times faster because the wrapper edits its native tree in C and skips a redundant de-duplication when a chain starts from a single node, where pyquery drives lxml under its jQuery-style wrapper.

How to migrate

Build a Query from a parsed document and call it with a selector; the method chains port almost name for name:

from turbohtml import parse
from turbohtml.query import Query

query = Query(parse("<div><a href='/u'>l</a><a>m</a></div>"))
print(query("a").filter("[href]").eq(0).add_class("seen").attr("href"))
print([anchor.text() for anchor in query("a").items()])
/u
['l', 'm']

pyquery

turbohtml

pq = PyQuery(html)

Query(parse(html))

pq("div.foo"), pq("a").find("span")

query("div.foo"), query("a").find("span")

.filter(sel), .eq(i), .closest(sel)

the same names

.attr("href"), .attr("k", "v")

the same names

.text(), .html()

the same names

.add_class(c), .remove_class(c), .toggle_class(c), .has_class(c)

add_class(), remove_class(), toggle_class(), has_class() (also on Query)

.parent(), .children(), .siblings()

the same names

iterating for item in pq("a").items()

for item in query("a").items()

jQuery pq("script").remove(), pq(".box b").remove()

node.remove("script"), node.remove(".box b")

jQuery $(".box b").contents().unwrap() (drop the tag, keep the text)

node.strip_tags(".box b")

pyquery’s .wrap_all(html) wraps a whole matched set in one new container in place; the node API has two methods for the shapes that fit a tree model cleanly. wrap_children() boxes every child of a container, and wrap_siblings() wraps a node and the contiguous run of siblings after it (through an until node, or to the last sibling), so query("p").wrap_all("<div/>") over a run of adjacent paragraphs becomes first.wrap_siblings(Element("div"), until=last):

pyquery

turbohtml

pq("section").contents().wrap_all("<div/>")

section.wrap_children(Element("div"))

pq(run).wrap_all("<div/>") over a contiguous run

first.wrap_siblings(Element("div"), until=last)

pyquery’s content setters – .html(markup) reparses a matched element’s children and .text(s) replaces them with one verbatim text node – map onto three element methods. set_inner_html() parses the markup as a fragment in the element’s context and replaces its children; set_text() replaces them with one verbatim text node; and insert_adjacent_html() splices a parsed fragment at a DOM position (the .append(markup) / insertAdjacentHTML shape):

pyquery

turbohtml

pq(el).html(markup)

el.set_inner_html(markup)

pq(el).text(s)

el.set_text(s)

pq(el).append(markup)

el.insert_adjacent_html("beforeend", markup)

Gotchas and pitfalls

  • .wrap_all over an arbitrary, non-contiguous set of nodes has no single node-method counterpart (the set has no shared anchor to place the wrapper at); wrap the contiguous run, or append() the scattered nodes into one new element and place it yourself.

  • pyquery’s network-fetching constructor (PyQuery(url=...)) is out of scope: fetch with httpx (or any client) and hand the bytes to turbohtml.parse().

  • pyquery exposes lxml’s .xpath(...) on the fluent wrapper itself; turbohtml’s Query is CSS-only, so an XPath chain drops to the node-level xpath() (XPath 1.0) or the find() grammar via Query.items.

  • turbohtml parses to the WHATWG spec, so a malformed document is fixed up exactly as a browser would (implied <tbody>, reparented <head> content); pyquery’s tree follows lxml’s HTML parser, which can differ on the same broken input.