From pyquery

pyquery latest releasepyquery supported Pythonspyquery licensepyquery monthly downloadspyquery total downloadspyquery GitHub starspyquery last commit

pyquery puts a jQuery-style fluent, chainable wrapper over lxml/cssselect, so you select and mutate a document with method chains. A single PyQuery object holds a matched set of nodes; calling it with a CSS selector, chaining .filter, .find, .eq, or .closest, and reading or writing attributes, text, HTML, and classes all return either a new wrapper or a scalar. It is a common pick for scraping and templating code that wants the DOM-manipulation feel of jQuery without a browser.

turbohtml covers that ground with turbohtml.query.Query, a fully type-annotated fluent wrapper whose selector, attribute, and class primitives run in the C extension over a native tree, so the same chaining idiom ports across with almost no rename and a large speed margin.

turbohtml vs pyquery

Dimension

turbohtml

pyquery

Scope

WHATWG HTML parser plus a jQuery-style Query wrapper over a native tree

jQuery-style wrapper only; parsing and the tree come from lxml

Feature breadth

CSS select/filter/traverse, attr/text/html/class ops, node-level mutation and XPath 1.0

CSS and XPath on the wrapper itself, broad jQuery method set, a network-fetching constructor

Performance

Selector and attribute primitives in C over a native tree; several to hundreds of times faster on the shared surface

Python wrapper delegating to lxml/cssselect

Typing

Fully type annotated with shipped stubs

No bundled type hints

Dependencies

Self-contained C extension, no Python runtime deps

Depends on lxml and cssselect

Maintenance

Actively developed alongside the parser

Mature, lightly maintained wrapper

Feature overlap

The shared surface ports one-to-one from a matched set built by calling the wrapper with a selector:

What turbohtml adds

  • A full WHATWG HTML parser in the same package, so parsing and querying share one native tree instead of delegating to lxml.

  • C-resident selector and attribute primitives, giving the large speed margin measured below.

  • Shipped type stubs for the whole surface, including Query and the node API.

  • No third-party runtime dependencies: the tree, selectors, and mutation all live in one self-contained C extension.

What pyquery has that turbohtml does not

  • A network-fetching constructor. PyQuery(url=...) fetches over HTTP for you. turbohtml has no equivalent; fetch with httpx (or any client) and hand the bytes to turbohtml.parse().

  • XPath on the fluent wrapper. pyquery exposes lxml’s .xpath(...) directly on a matched set. turbohtml’s Query is CSS-only; drop to the node-level xpath() (XPath 1.0) or the find() grammar via Query.items.

  • ``.wrap_all`` over a non-contiguous set. pyquery wraps any matched set in one container. turbohtml’s node API covers the tree-clean shapes (see below) but has no counterpart for an arbitrary scattered set; append() the nodes into a new element and place it yourself.

Performance

operation

turbohtml

pyquery

parse to a tree — wpt tiny (0.6 kB)

1.21 µs

6.96 µs (5.8x)

parse to a tree — wpt small (4 kB)

9.58 µs

31.3 µs (3.3x)

parse to a tree — wpt medium (9.6 kB)

24.1 µs

77.9 µs (3.3x)

parse to a tree — wpt large (92 kB)

209 µs

877 µs (4.2x)

parse to a tree — wpt CJK (124 kB)

408 µs

1.44 ms (3.6x)

parse to a tree — whatwg spec (235 kB)

400 µs

1.66 ms (4.2x)

find every anchor — daring fireball (10 kB)

371 ns

10.2 µs (27.4x)

find every anchor — ars technica (56 kB)

817 ns

9.8 µs (12.0x)

find every anchor — mozilla blog (95 kB)

1.16 µs

31.8 µs (27.4x)

find every anchor — whatwg spec (235 kB)

1.36 µs

60.1 µs (44.3x)

select div a[href] — daring fireball (10 kB)

610 ns

29.6 µs (48.6x)

select div a[href] — ars technica (56 kB)

1.47 µs

17.2 µs (11.8x)

select div a[href] — mozilla blog (95 kB)

2.1 µs

810 µs (386x)

select div a[href] — whatwg spec (235 kB)

1.78 µs

1.42 ms (796x)

match each anchor against div a[href] — daring fireball (10 kB)

1.79 µs

432 µs (242x)

match each anchor against div a[href] — ars technica (56 kB)

4.21 µs

10.4 µs (2.5x)

match each anchor against div a[href] — mozilla blog (95 kB)

5.76 µs

1.41 ms (245x)

match each anchor against div a[href] — whatwg spec (235 kB)

6.68 µs

1.69 ms (253x)

find by text content — daring fireball (10 kB)

24.4 µs

51.4 µs (2.2x)

find by text content — ars technica (56 kB)

178 µs

239 µs (1.4x)

find by text content — mozilla blog (95 kB)

287 µs

440 µs (1.6x)

find by text content — whatwg spec (235 kB)

560 µs

1.18 ms (2.2x)

collect visible text — daring fireball (10 kB)

2.7 µs

128 µs (47.4x)

collect visible text — ars technica (56 kB)

13 µs

10 µs (0.8x)

collect visible text — mozilla blog (95 kB)

22.4 µs

913 µs (40.9x)

collect visible text — whatwg spec (235 kB)

75.4 µs

4.85 ms (64.4x)

serialize a parsed tree — daring fireball (10 kB)

7.02 µs

16.9 µs (2.5x)

serialize a parsed tree — ars technica (56 kB)

38.8 µs

75.3 µs (2.0x)

serialize a parsed tree — mozilla blog (95 kB)

76.3 µs

159 µs (2.1x)

serialize a parsed tree — whatwg spec (235 kB)

195 µs

379 µs (2.0x)

tag every link rel=nofollow — daring fireball (10 kB)

3.32 µs

19 µs (5.8x)

tag every link rel=nofollow — ars technica (56 kB)

13.1 µs

56.3 µs (4.4x)

tag every link rel=nofollow — mozilla blog (95 kB)

20.9 µs

57.2 µs (2.8x)

tag every link rel=nofollow — whatwg spec (235 kB)

42.6 µs

308 µs (7.3x)

class add/remove on every link — daring fireball (10 kB)

2.24 µs

43.5 µs (19.4x)

class add/remove on every link — ars technica (56 kB)

8.89 µs

10 µs (1.2x)

class add/remove on every link — mozilla blog (95 kB)

8.78 µs

151 µs (17.2x)

class add/remove on every link — whatwg spec (235 kB)

8.7 µs

187 µs (21.5x)

drop tags with content (remove) — daring fireball (10 kB)

23.8 µs

155 µs (6.5x)

drop tags with content (remove) — ars technica (56 kB)

115 µs

323 µs (2.8x)

drop tags with content (remove) — mozilla blog (95 kB)

260 µs

1.12 ms (4.4x)

drop tags with content (remove) — whatwg spec (235 kB)

645 µs

2.62 ms (4.1x)

unwrap tags keep content (strip_tags) — daring fireball (10 kB)

24.3 µs

175 µs (7.3x)

unwrap tags keep content (strip_tags) — ars technica (56 kB)

122 µs

323 µs (2.7x)

unwrap tags keep content (strip_tags) — mozilla blog (95 kB)

264 µs

1.18 ms (4.5x)

unwrap tags keep content (strip_tags) — whatwg spec (235 kB)

683 µs

3.06 ms (4.5x)

replace body inner HTML — daring fireball (10 kB)

2.12 µs

18.1 µs (8.6x)

replace body inner HTML — ars technica (56 kB)

8.69 µs

13.3 µs (1.6x)

replace body inner HTML — mozilla blog (95 kB)

14.4 µs

123 µs (8.6x)

replace body inner HTML — whatwg spec (235 kB)

40.1 µs

370 µs (9.3x)

replace body text — daring fireball (10 kB)

1.18 µs

15.4 µs (13.1x)

replace body text — ars technica (56 kB)

7.73 µs

13.9 µs (1.8x)

replace body text — mozilla blog (95 kB)

12.9 µs

122 µs (9.5x)

replace body text — whatwg spec (235 kB)

40.6 µs

365 µs (9.0x)

walk every descendant — daring fireball (10 kB)

3.26 µs

76.7 µs (23.6x)

walk every descendant — ars technica (56 kB)

13.4 µs

227 µs (17.0x)

walk every descendant — mozilla blog (95 kB)

28 µs

619 µs (22.1x)

walk every descendant — whatwg spec (235 kB)

96.8 µs

2.15 ms (22.2x)

fluent jQuery-style chain — daring fireball (10 kB)

3.69 µs

87.5 µs (23.8x)

fluent jQuery-style chain — ars technica (56 kB)

11.3 µs

22.8 µs (2.1x)

fluent jQuery-style chain — mozilla blog (95 kB)

16.9 µs

245 µs (14.5x)

fluent jQuery-style chain — whatwg spec (235 kB)

29 µs

313 µs (10.9x)

extract every link — daring fireball (10 kB)

8.49 µs

167 µs (19.7x)

extract every link — ars technica (56 kB)

27.6 µs

11.5 µs (0.5x)

extract every link — mozilla blog (95 kB)

50.1 µs

598 µs (12.0x)

extract every link — whatwg spec (235 kB)

84.6 µs

719 µs (8.5x)

absolutize every link — daring fireball (10 kB)

82.6 µs

386 µs (4.7x)

absolutize every link — ars technica (56 kB)

212 µs

13 µs (0.1x)

absolutize every link — mozilla blog (95 kB)

482 µs

1.38 ms (2.9x)

absolutize every link — whatwg spec (235 kB)

270 µs

1.69 ms (6.3x)

rewrite every link — daring fireball (10 kB)

4.69 µs

299 µs (63.8x)

rewrite every link — ars technica (56 kB)

11.5 µs

11.1 µs (1.0x)

rewrite every link — mozilla blog (95 kB)

21.7 µs

999 µs (46.2x)

rewrite every link — whatwg spec (235 kB)

38.1 µs

1.16 ms (30.4x)

social-card extraction — head

1.79 µs

73 µs (40.7x)

social-card extraction — article 8 KiB

22.1 µs

163 µs (7.4x)

extract @href per match — daring fireball (10 kB)

2.49 µs

162 µs (65.1x)

extract @href per match — ars technica (56 kB)

6.29 µs

10.5 µs (1.7x)

extract @href per match — mozilla blog (95 kB)

8.51 µs

756 µs (88.9x)

extract @href per match — whatwg spec (235 kB)

9.05 µs

649 µs (71.8x)

extract text per match — daring fireball (10 kB)

2.75 µs

92 µs (33.5x)

extract text per match — ars technica (56 kB)

6.39 µs

10.3 µs (1.7x)

extract text per match — mozilla blog (95 kB)

10.3 µs

345 µs (33.7x)

extract text per match — whatwg spec (235 kB)

10.8 µs

384 µs (35.5x)

extract URL hints — base_url / get_base_url

1.27 µs

18.4 µs (14.6x)

extract URL hints — meta_refresh / get_meta_refresh

1.29 µs

26.6 µs (20.7x)

extract filtered page links — daring fireball (10 kB)

127 µs

167 µs (1.4x)

extract filtered page links — ars technica (56 kB)

310 µs

10.5 µs (0.1x)

extract filtered page links — mozilla blog (95 kB)

514 µs

541 µs (1.1x)

extract filtered page links — whatwg spec (235 kB)

889 µs

631 µs (0.8x)

The whole shared surface – chaining a select/filter/read, setting content, bulk-editing tags, and reading a value off every match – runs several to hundreds of times faster because the wrapper edits its native tree in C and skips a redundant de-duplication when a chain starts from a single node, where pyquery drives lxml under its jQuery-style wrapper.

How to migrate

Build a Query from a parsed document and call it with a selector; the method chains port almost name for name:

from turbohtml import parse
from turbohtml.query import Query

query = Query(parse("<div><a href='/u'>l</a><a>m</a></div>"))
print(query("a").filter("[href]").eq(0).add_class("seen").attr("href"))
print([anchor.text() for anchor in query("a").items()])
/u
['l', 'm']

pyquery

turbohtml

pq = PyQuery(html)

Query(parse(html))

pq("div.foo"), pq("a").find("span")

query("div.foo"), query("a").find("span")

.filter(sel), .eq(i), .closest(sel)

the same names

.attr("href"), .attr("k", "v")

the same names

.text(), .html()

the same names

.add_class(c), .remove_class(c), .toggle_class(c), .has_class(c)

add_class(), remove_class(), toggle_class(), has_class() (also on Query)

.parent(), .children(), .siblings()

the same names

iterating for item in pq("a").items()

for item in query("a").items()

jQuery pq("script").remove(), pq(".box b").remove()

node.remove("script"), node.remove(".box b")

jQuery $(".box b").contents().unwrap() (drop the tag, keep the text)

node.strip_tags(".box b")

pyquery’s .wrap_all(html) wraps a whole matched set in one new container in place; the node API has two methods for the shapes that fit a tree model cleanly. wrap_children() boxes every child of a container, and wrap_siblings() wraps a node and the contiguous run of siblings after it (through an until node, or to the last sibling), so query("p").wrap_all("<div/>") over a run of adjacent paragraphs becomes first.wrap_siblings(Element("div"), until=last):

pyquery

turbohtml

pq("section").contents().wrap_all("<div/>")

section.wrap_children(Element("div"))

pq(run).wrap_all("<div/>") over a contiguous run

first.wrap_siblings(Element("div"), until=last)

pyquery’s content setters – .html(markup) reparses a matched element’s children and .text(s) replaces them with one verbatim text node – map onto three element methods. set_inner_html() parses the markup as a fragment in the element’s context and replaces its children; set_text() replaces them with one verbatim text node; and insert_adjacent_html() splices a parsed fragment at a DOM position (the .append(markup) / insertAdjacentHTML shape):

pyquery

turbohtml

pq(el).html(markup)

el.set_inner_html(markup)

pq(el).text(s)

el.set_text(s)

pq(el).append(markup)

el.insert_adjacent_html("beforeend", markup)

Gotchas and pitfalls

  • .wrap_all over an arbitrary, non-contiguous set of nodes has no single node-method counterpart (the set has no shared anchor to place the wrapper at); wrap the contiguous run, or append() the scattered nodes into one new element and place it yourself.

  • pyquery’s network-fetching constructor (PyQuery(url=...)) is out of scope: fetch with httpx (or any client) and hand the bytes to turbohtml.parse().

  • pyquery exposes lxml’s .xpath(...) on the fluent wrapper itself; turbohtml’s Query is CSS-only, so an XPath chain drops to the node-level xpath() (XPath 1.0) or the find() grammar via Query.items.

  • turbohtml parses to the WHATWG spec, so a malformed document is fixed up exactly as a browser would (implied <tbody>, reparented <head> content); pyquery’s tree follows lxml’s HTML parser, which can differ on the same broken input.