From selectolax

selectolax latest releaseselectolax supported Pythonsselectolax licenseselectolax monthly downloadsselectolax total downloadsselectolax GitHub starsselectolax last commit

selectolax is a fast HTML parser that wraps a C engine and exposes CSS selection. It ships two backends: HTMLParser over Modest and LexborHTMLParser over lexbor. You query with CSS selectors (css / css_first), read text through the text() method, reach attributes via node.attributes, and mutate the tree with a small set of operations (decompose, unwrap, strip_tags, unwrap_tags). It has no XPath and no regex extraction. Its niche is high-throughput web scraping and data extraction where CSS selection over a compiled C tree is the whole job.

turbohtml covers the same ground with a single native, spec-compliant HTML5 engine: select() / select_one() for CSS, text as a property, attrs for attributes, and the same drop/unwrap operations. On top of that surface it adds the find / find_all filter grammar, XPath, regex extraction, a full mutation surface, and markdown/text/minify output, all fully typed.

turbohtml vs selectolax

Dimension

turbohtml

selectolax

Scope

Spec-compliant HTML5 parser plus native CSS/XPath selection, regex extraction, mutation, and serialization

CSS-only selection and light tree editing over a bundled C engine (Modest or lexbor); no XPath, no regex

Feature breadth

CSS Selectors Level 4, the find/find_all filter grammar, XPath, regex, DOM mutation, and markdown/text/minify/sanitize output

CSS selection, text() extraction, attribute access, and a handful of drop/unwrap/decompose operations

Performance

Compiles a selector once and matches by interned integer atoms; collects text in one C pass; see the table below

Fast CSS matching in C, but text collection and node access cross the C boundary per node

Typing

Fully typed, ships .pyi stubs; queries return typed Node

Typed API surface, but node access is string-centric and text is a method call

Dependencies

Self-contained C extension, no runtime dependencies

Self-contained C extension (Modest/lexbor bundled), no runtime dependencies

Maintenance

Actively developed

Actively maintained

Feature overlap

These map 1:1 and port with a rename:

  • CSS selection: node.css(sel) / node.css_first(sel) become select() / select_one().

  • Selector test: node.css_matches(sel) becomes matches().

  • Tag name: node.tag stays tag.

  • Attributes: node.attributes becomes attrs, and a single value reads through attr().

  • Outer HTML: node.html stays html.

  • Node removal that keeps the children: node.unwrap() stays unwrap().

  • Node removal with its subtree: node.decompose() stays decompose().

  • Bulk tag stripping: parser.strip_tags([...]) (drop tags with content) becomes remove(), and node.unwrap_tags([...]) (keep content) becomes strip_tags(). Both turbohtml methods take a full CSS selector, not a tag-name list.

What turbohtml adds

What selectolax has that turbohtml does not

  • A choice of parser backends (HTMLParser over Modest vs LexborHTMLParser over lexbor): no equivalent. turbohtml is a single native engine; there is one tree builder and no backend switch.

  • text(deep=..., separator=..., strip=...) shaping text extraction in one call: no exact equivalent. Read text for the flat string, iterate stripped_strings and join with your own separator, or call to_text() for a formatted rendering.

  • The bundled engine’s raw C-level node handles and lexbor-specific knobs: not exposed. turbohtml’s public surface is the typed Python tree, not the underlying engine’s C API.

Performance

turbohtml’s lighter native tree parses, selects, and serializes faster than selectolax’s heavier object layer over lexbor. It drops a set of tags with their subtrees faster (remove() against strip_tags, over a 92 kB page of 839 <code>/<a>/<q> elements), tests a compiled selector against every anchor (matches() against css_matches) 35 to 45 times faster, and collects a node’s visible text (text against selectolax’s text() method) seven to nine times faster, concatenating in one C pass where selectolax crosses the lexbor boundary per node:

operation

turbohtml

selectolax

parse to a tree — wpt tiny (0.6 kB)

1.53 µs

9.51 µs (6.3x ±6%)

parse to a tree — wpt small (4 kB)

12.2 µs

56 µs (4.6x ±3%)

parse to a tree — wpt medium (9.6 kB)

32.5 µs

137 µs (4.3x ±6%)

parse to a tree — wpt large (92 kB)

276 µs

1.26 ms (4.6x ±2%)

parse to a tree — wpt CJK (124 kB)

532 µs

2.96 ms (5.6x ±5%)

parse to a tree — whatwg spec (235 kB)

519 µs

2.45 ms (4.8x ±3%)

parse to a tree — common tags (13 kB)

78.5 µs

333 µs (4.3x ±3%)

find every anchor — daring fireball (10 kB)

372 ns

5.67 µs (15.3x ±2%)

find every anchor — ars technica (56 kB)

811 ns

15.9 µs (19.7x ±2%)

find every anchor — mozilla blog (95 kB)

1.12 µs

28.5 µs (25.5x ±3%)

find every anchor — whatwg spec (235 kB)

1.3 µs

75.3 µs (58.0x ±2%)

select div a[href] — daring fireball (10 kB)

595 ns

7.58 µs (12.8x ±1%)

select div a[href] — ars technica (56 kB)

1.43 µs

20.2 µs (14.1x ±2%)

select div a[href] — mozilla blog (95 kB)

2.01 µs

34.1 µs (17.0x ±1%)

select div a[href] — whatwg spec (235 kB)

1.76 µs

79.4 µs (45.1x ±1%)

select div:has(a) — daring fireball (10 kB)

262 ns

4.95 µs (19.0x ±2%)

select div:has(a) — ars technica (56 kB)

1.26 µs

17.8 µs (14.1x ±2%)

select div:has(a) — mozilla blog (95 kB)

8.54 µs

51.9 µs (6.1x ±2%)

select div:has(a) — whatwg spec (235 kB)

5.54 µs

82.7 µs (15.0x ±2%)

match each anchor against div a[href] — daring fireball (10 kB)

1.85 µs

62.4 µs (33.8x ±3%)

match each anchor against div a[href] — ars technica (56 kB)

4.23 µs

152 µs (36.0x ±2%)

match each anchor against div a[href] — mozilla blog (95 kB)

5.83 µs

213 µs (36.6x ±1%)

match each anchor against div a[href] — whatwg spec (235 kB)

6.46 µs

287 µs (44.5x ±2%)

collect visible text — daring fireball (10 kB)

2.63 µs

21.4 µs (8.2x ±2%)

collect visible text — ars technica (56 kB)

13.5 µs

89.8 µs (6.7x ±1%)

collect visible text — mozilla blog (95 kB)

21.9 µs

203 µs (9.3x ±2%)

collect visible text — whatwg spec (235 kB)

83.7 µs

756 µs (9.1x ±1%)

serialize a parsed tree — daring fireball (10 kB)

6.52 µs

28.6 µs (4.4x ±2%)

serialize a parsed tree — ars technica (56 kB)

34.8 µs

154 µs (4.5x ±1%)

serialize a parsed tree — mozilla blog (95 kB)

67.4 µs

307 µs (4.6x ±1%)

serialize a parsed tree — whatwg spec (235 kB)

174 µs

739 µs (4.3x ±1%)

tag every link rel=nofollow — daring fireball (10 kB)

4.74 µs

16 µs (3.4x ±3%)

tag every link rel=nofollow — ars technica (56 kB)

16.6 µs

39.1 µs (2.4x ±4%)

tag every link rel=nofollow — mozilla blog (95 kB)

25.4 µs

63.6 µs (2.6x ±5%)

tag every link rel=nofollow — whatwg spec (235 kB)

51.4 µs

118 µs (2.4x ±2%)

class add/remove on every link — daring fireball (10 kB)

2.62 µs

26.6 µs (10.2x ±2%)

class add/remove on every link — ars technica (56 kB)

9.67 µs

73.2 µs (7.6x ±2%)

class add/remove on every link — mozilla blog (95 kB)

10.3 µs

102 µs (10.0x ±2%)

class add/remove on every link — whatwg spec (235 kB)

10.1 µs

156 µs (15.5x ±2%)

drop tags with content (remove) — daring fireball (10 kB)

23.2 µs

98.5 µs (4.3x ±1%)

drop tags with content (remove) — ars technica (56 kB)

112 µs

480 µs (4.3x ±3%)

drop tags with content (remove) — mozilla blog (95 kB)

252 µs

1.53 ms (6.1x ±3%)

drop tags with content (remove) — whatwg spec (235 kB)

650 µs

2.77 ms (4.3x ±3%)

unwrap tags keep content (strip_tags) — daring fireball (10 kB)

24.4 µs

113 µs (4.7x ±2%)

unwrap tags keep content (strip_tags) — ars technica (56 kB)

120 µs

521 µs (4.4x ±1%)

unwrap tags keep content (strip_tags) — mozilla blog (95 kB)

261 µs

1.61 ms (6.2x ±2%)

unwrap tags keep content (strip_tags) — whatwg spec (235 kB)

669 µs

2.93 ms (4.4x ±2%)

walk every descendant — daring fireball (10 kB)

3.37 µs

26.3 µs (7.8x ±36%)

walk every descendant — ars technica (56 kB)

13.3 µs

69.1 µs (5.2x ±4%)

walk every descendant — mozilla blog (95 kB)

28.5 µs

152 µs (5.4x ±1%)

walk every descendant — whatwg spec (235 kB)

97.5 µs

518 µs (5.4x ±2%)

extract every link — daring fireball (10 kB)

13.5 µs

21.6 µs (1.7x ±21%)

extract every link — ars technica (56 kB)

45.7 µs

70.2 µs (1.6x ±17%)

extract every link — mozilla blog (95 kB)

75.5 µs

110 µs (1.5x ±15%)

extract every link — whatwg spec (235 kB)

66.4 µs

179 µs (2.7x ±25%)

absolutize every link — daring fireball (10 kB)

39.4 µs

77.9 µs (2.0x ±4%)

absolutize every link — ars technica (56 kB)

91.8 µs

202 µs (2.2x ±2%)

absolutize every link — mozilla blog (95 kB)

157 µs

298 µs (1.9x ±5%)

absolutize every link — whatwg spec (235 kB)

224 µs

347 µs (1.6x ±3%)

rewrite every link — daring fireball (10 kB)

3 µs

25.4 µs (8.5x ±17%)

rewrite every link — ars technica (56 kB)

10.7 µs

68.9 µs (6.5x ±34%)

rewrite every link — mozilla blog (95 kB)

19.9 µs

91.2 µs (4.6x ±3%)

rewrite every link — whatwg spec (235 kB)

33.3 µs

149 µs (4.5x ±3%)

social-card extraction — head

1.83 µs

15 µs (8.2x ±2%)

social-card extraction — article 8 KiB

21.9 µs

134 µs (6.2x ±1%)

extract @href per match — daring fireball (10 kB)

3.57 µs

19 µs (5.4x ±7%)

extract @href per match — ars technica (56 kB)

9.06 µs

66.5 µs (7.4x ±4%)

extract @href per match — mozilla blog (95 kB)

12.5 µs

102 µs (8.3x ±4%)

extract @href per match — whatwg spec (235 kB)

14.4 µs

149 µs (10.4x ±12%)

extract text per match — daring fireball (10 kB)

3.44 µs

22.7 µs (6.6x ±5%)

extract text per match — ars technica (56 kB)

8.73 µs

59.3 µs (6.8x ±5%)

extract text per match — mozilla blog (95 kB)

14.6 µs

104 µs (7.2x ±16%)

extract text per match — whatwg spec (235 kB)

14.6 µs

166 µs (11.4x ±7%)

extract URL hints — base_url / get_base_url

1.17 µs

8.43 µs (7.3x ±3%)

extract URL hints — meta_refresh / get_meta_refresh

1.21 µs

8.85 µs (7.4x ±2%)

extract filtered page links — daring fireball (10 kB)

144 µs

222 µs (1.6x ±16%)

extract filtered page links — ars technica (56 kB)

363 µs

772 µs (2.2x ±9%)

extract filtered page links — mozilla blog (95 kB)

619 µs

2.12 ms (3.5x ±10%)

extract filtered page links — whatwg spec (235 kB)

1.06 ms

3.03 ms (2.9x ±13%)

How to migrate

Replace LexborHTMLParser(html) (or HTMLParser(html)) with turbohtml.parse(), then swap css for select() and drop the parentheses on text.

selectolax

turbohtml

LexborHTMLParser(html)

turbohtml.parse()

parser.root, parser.body

doc.root, doc.find("body")

node.css("a"), node.css_first("a")

select(), select_one()

node.css_matches("a")

matches()

node.tag

tag (same)

node.attributes

attrs, attr()

node.text() (a method)

text (a property), strings, stripped_strings

node.html, node.decompose(), node.unwrap()

html, decompose(), unwrap()

parser.strip_tags(["script"]), node.unwrap_tags(["b"])

node.remove("script"), node.strip_tags("b")

doc = parse("<ul><li>a</li><li>b</li></ul>")
print([li.text for li in doc.select("li")])
['a', 'b']

Gotchas and pitfalls

  • node.text is a property in turbohtml; drop the parentheses. selectolax’s text(deep=..., separator=..., strip=...) keywords have no single-call equivalent: use stripped_strings with your own join for a separator, or to_text() for formatted output.

  • The bulk tag strippers are named the other way around: selectolax’s strip_tags drops the tags with their content (turbohtml’s remove()), while its unwrap_tags keeps the content (turbohtml’s strip_tags()). Both turbohtml methods take a full CSS selector, not a tag-name list.

  • selectolax queries are CSS-only; there is no xpath or re to port. Where you would have chained several css calls, turbohtml’s find / find_all filter grammar, xpath(), and re() cover the same intent in one call.

  • css_first returns None on a miss; so does select_one(), and select() returns an empty list, so guard the None before reading .text or attr().

  • selectolax’s lexbor-specific knobs, its Modest-vs-lexbor backend choice, and its raw C-level node handles are not exposed by turbohtml; the public surface is the typed Python tree, not the underlying engine’s C API.