From cssselect¶
cssselect translates a CSS selector into an equivalent XPath 1.0 expression. It
parses the selector in Python, walks the parsed tree, and emits an XPath string; it does not itself match against a
document. That translator is the engine behind lxml.cssselect(), parsel (and so
Scrapy), and pyquery, wherever a library wants CSS syntax on top of an XPath
evaluator. Its public surface is small: a GenericTranslator (XML rules) and an HTMLTranslator (HTML lowercasing
rules), each exposing css_to_xpath(css, prefix="descendant-or-self::"), plus its SelectorError /
SelectorSyntaxError / ExpressionError error types.
turbohtml.convert.css_to_xpath() does the same job in a single C pass over the parsed selector, and
turbohtml.convert.GenericTranslator / turbohtml.convert.HTMLTranslator keep cssselect’s
translator-object shape and error names so a port is mechanical. Because turbohtml runs a CSS selector engine and an
XPath engine in the same process, the emitted XPath is validated differentially against the native CSS engine rather
than trusted on its own.
turbohtml vs cssselect¶
Dimension |
turbohtml |
cssselect |
|---|---|---|
Scope |
CSS-to-XPath translation as one feature of a full HTML5 parser, CSS/XPath query engine, and serializer |
CSS-to-XPath translation only; matching is left to the host XPath evaluator |
Feature breadth |
Selectors 4 pseudo-classes with full complex arguments in |
Selectors 3 plus documented approximations; several cases carry |
Performance |
C translator; 4x faster on a bare type selector, 36x-45x on realistic selectors (see below) |
Pure-Python tokenizer and parser dominate on anything past a trivial selector |
Typing |
Fully typed, ships |
No inline type annotations; third-party stubs only |
Dependencies |
None (self-contained C extension) |
None at runtime |
Maintenance |
Actively developed as part of turbohtml |
Stable and maintained by the Scrapy project, low change rate |
Feature overlap¶
Portable 1:1 without behavior change:
GenericTranslator().css_to_xpath(css)andHTMLTranslator().css_to_xpath(css)— same method name and signature, including the positionalprefixsecond argument.The default
prefix="descendant-or-self::"and its role of scoping each translated arm to the context node’s subtree.A comma-separated selector list translating to an XPath
|union, one arm per selector.The two failure modes as typed errors:
turbohtml.SelectorSyntaxErrorfor a selector the grammar rejects andExpressionErrorfor a valid selector with no XPath 1.0 form.
What turbohtml adds¶
A plain
turbohtml.convert.css_to_xpath()function, so callers who never wanted a translator object can skip it.Full complex selectors inside logical pseudo-classes:
:is(nav a),:where(...),:not(...), and the relative:has(> a)/:has(+ a)/:has(~ a)forms.WHATWG form-state pseudo-classes —
:disabled/:enabled/:checked/:required/:optional/:read-only/:read-write— including thefieldsetfirst-legendexemption cssselect marks as aFIXME, and:nth-child(An+B of S).Selectors 4
:empty(whitespace-only elements match) and the WHATWG case-insensitive attribute set (type,lang,rel, …) compared case-insensitively, as browsers do.Context-free output: every emitted predicate avoids bare
position()tests, so a translated fragment keeps its meaning when embedded in a larger expression.The native CSS engine itself (
turbohtml.Node.select()), so translation to XPath is optional rather than the only way to run a selector.
What cssselect has that turbohtml does not¶
A true XML / case-sensitive translation mode. cssselect’s
GenericTranslatormatches element and attribute names case-sensitively; turbohtml always applies the HTML lowercasing rules.HTMLTranslator(xhtml=True)accepts and records the flag for signature compatibility but does not change the output. No equivalent for genuinely case-sensitive XML selection.The non-standard cssselect extensions
:contains()and[attr!=value]. These are not CSS and do not parse in turbohtml; they raiseSelectorSyntaxError. Workaround: use the XPathcontains(., ...)predicate directly, or:not([attr=value])for negated attribute matching.:scopeanywhere other than the leftmost compound. turbohtml translates:scope > divbut raisesExpressionErrorfor:scopedeeper in a selector, matching cssselect’s own leftmost-only behavior; neither is more general here.
Performance¶
The C translator is 4x faster than cssselect on a bare type selector and 36x to 45x faster on realistic selectors, where cssselect’s Python tokenizer dominates. Each ratio is against turbohtml:
CSS selector to XPath 1.0 |
turbohtml |
cssselect |
|---|---|---|
type |
264 ns |
2.17 µs (8.3x) |
compound |
361 ns |
12.8 µs (35.5x) |
structural |
325 ns |
12.1 µs (37.4x) |
complex |
534 ns |
23 µs (43.2x) |
group |
565 ns |
15 µs (26.6x) |
How to migrate¶
Swap the import; cssselect’s HTMLTranslator / GenericTranslator live under turbohtml.convert with the
same method, and a bare css_to_xpath() function is available for call sites that never needed a
translator object.
cssselect |
turbohtml |
|---|---|
|
|
|
|
|
|
|
|
|
|
# cssselect
from cssselect import HTMLTranslator
xpath = HTMLTranslator().css_to_xpath("ul > li.item")
# turbohtml, either shape
from turbohtml.convert import HTMLTranslator, css_to_xpath
xpath = HTMLTranslator().css_to_xpath("ul > li.item")
xpath = css_to_xpath("ul > li.item")
from turbohtml.convert import css_to_xpath
print(css_to_xpath("div#main a[href^='https']"))
descendant-or-self::div[@id = 'main']/descendant::a[starts-with(@href, 'https')]
The prefix argument works as in cssselect (default descendant-or-self::). A selector the grammar rejects raises
turbohtml.SelectorSyntaxError – the one error every turbohtml selector-parse path shares, a
ValueError – and a valid selector with no XPath 1.0 form raises ExpressionError, under the
cssselect-shaped SelectorError.
from turbohtml.convert import ExpressionError, SelectorSyntaxError, css_to_xpath
try:
css_to_xpath("li:")
except SelectorSyntaxError as error:
print(error)
try:
css_to_xpath(":dir(rtl)")
except ExpressionError as error:
print(error)
invalid CSS selector "li:": expected an identifier at position 3
:dir() cannot be expressed in XPath 1.0
Gotchas and pitfalls¶
The emitted string differs from cssselect’s; only the selected node-set is the contract. Compare results rather than expression text.
Both translator classes apply the HTML rules (element and attribute names lowercase). turbohtml has no XML (case-sensitive) mode, so
HTMLTranslator(xhtml=True)only records the flag.:scopetranslates only as the leftmost compound (:scope > div), matching how cssselect’s own translation behaves; anywhere else raisesExpressionError.cssselect’s non-standard extensions
:contains()and[attr!=value]are not CSS and do not parse; they raiseSelectorSyntaxError. Use the XPathcontains(., ...)predicate or:not([attr=value])directly.[lang|="en"],[type=CHECKBOX],li:emptyon whitespace-only elements, and:disabledon hidden inputs or aroundlegendselect per the current specs, so their node-sets can differ from cssselect’s approximations (the differential suite pins each divergence).SelectorSyntaxErroristurbohtml.SelectorSyntaxError, aValueError(not cssselect’sSyntaxError), unifying it with the CSS matching engine;ExpressionErrorsubclassesRuntimeErrorunderSelectorError. Code catching cssselect’s classes by import must switch the import toturbohtml.convert.