Nodes¶

A parsed tree is made of nodes. Node carries the navigation, query, mutation, and serialization surface every node shares; the concrete types below add their own data. Text is a real Text child node (the WHATWG DOM shape), so there is no text/tail split.

class turbohtml.Node¶

Common navigation shared by every node in a parsed tree. Text appears as real child nodes (the WHATWG DOM shape), so there is no text/tail split.

ancestors: Iterator[Element | Document]¶: an iterator from parent up to the document

article() → Article¶: Return an Article record for the dominant content under this node: the scored content body (element), its layout-aware plain text (text, as main_text()), and the page metadata harvested from the document – title, byline, date, description, lang, canonical, site_name, tags and image. element is None and text is empty when nothing reads as content; each single-valued metadata field is None when absent and tags is an empty tuple. Title comes from <h1>, then og:title, then <title>; byline from a rel=author link, then a meta author, then article:author; date from <time>, then article:published_time, then a common date meta; description from og:description, then a meta description; lang from <html lang>; canonical from <link rel=canonical>, then og:url; site_name from og:site_name, then a meta application-name; tags from every <meta name=keywords> (comma-split) and article:tag; image from og:image, then twitter:image. Pure C.

assigned_slot: Element | None¶: the <slot> element this node is assigned to in its shadow host, or None when it is unassigned or the host’s shadow root is closed

canonicalize(options: Canonical | None = None) → bytes¶

Serialize this node and its subtree to Canonical XML (c14n), the byte-exact form an XML signature signs, returned as UTF-8 bytes.

Attributes are reordered (namespace declarations first, then each attribute by namespace URI and local name), redundant namespace declarations are dropped, empty elements are written as start-end pairs, and the c14n character-reference rules apply. The whole subtree is canonicalized, so c14n 1.0 and 1.1 coincide here bar the apex’s inherited xml: attributes.

Parameters:: options – a Canonical configuration object (version, exclusive, with_comments, inclusive_ns_prefixes), or None for the defaults.
Returns:: the canonical XML, encoded as UTF-8 bytes.
Raises:: TypeError – if options is not a Canonical configuration object.

children: Sequence[Node]¶: the child nodes as a tuple

closest(selector: str, /) → Element | None¶

Find the nearest Element matching a CSS selector, testing this node then each ancestor.

Parameters:: selector – the CSS selector.
Returns:: the nearest matching Element, or None.
Raises:: ValueError – the selector is not valid CSS; the message names the reason and the source offset.

decompose() → None¶: Detach this node and its subtree from the document and drop it.

descendants: Iterator[Node]¶: an iterator over every descendant in document order

encode(encoding: str = 'utf-8', options: Html | None = None) → bytes¶

Serialize this node and its subtree to bytes, with the same formatting controls as serialize().

Parameters:

encoding – the codec to encode the markup with.
options – an Html configuration object, or None for the defaults.

Returns:

the serialized markup encoded as bytes.

Raises:

TypeError – if options is not an Html configuration object.
LookupError – if encoding names a codec Python does not know.
UnicodeEncodeError – if the markup has characters the encoding cannot represent.

equals(other: Node, /) → bool¶

Test whether this node and other are structurally equal: the same node type, and for an element the same tag (namespace-aware) and attributes (names and values, order-independent per the DOM) with the same children compared in order; for a Text/Comment/other leaf the same data. The two nodes may come from different documents. This is the BeautifulSoup notion of tree equality.

Distinct from ==, which is node identity: two separate parses of the same markup are ==-unequal but equals()-equal.

Parameters:: other – the node to compare against.
Returns:: whether the two subtrees are structurally equal.
Raises:: TypeError – if other is not a node.

extract() → Node¶

Detach this node from its parent, leaving a standalone node the caller can reinsert elsewhere.

Returns:: this node, detached.

Find the first Element along axis matching the tag filter and every attribute filter. A filter is a str, bool, compiled regex, callable, or a list of those.

Parameters:

tag – filter on the tag name.
axis – which nodes to walk relative to this one.
attrs – a mapping of attribute name to filter.
class_ – filter on a token of the class attribute.
text – match the element’s collected text (an exact str, a regex search, or a callable predicate); filter a literal text attribute through attrs={‘text’: …}.
filters – further attribute filters given as keyword arguments.

Returns:

the first matching Element, or None.

Raises:

TypeError – if axis is not an Axis, or a filter is not a str, bool, regex, callable, or a list of those.

Find every Element along axis matching the tag filter and every attribute filter, with the same filter forms as find().

Parameters:

tag – filter on the tag name.
axis – which nodes to walk relative to this one.
attrs – a mapping of attribute name to filter.
class_ – filter on a token of the class attribute.
text – match each element’s collected text (an exact str, a regex search, or a callable predicate).
limit – stop after this many matches; None collects them all.
filters – further attribute filters given as keyword arguments.

Returns:

the matching Elements in document order.

Raises:

TypeError – if axis is not an Axis, limit is not an int or None, or a filter is not a str, bool, regex, callable, or a list of those.
ValueError – if limit is negative.

flattened_children: list[Node]¶: the flattened-tree children as a list: a shadow host yields its shadow tree with each <slot> replaced by its assigned nodes (or fallback content)

following: Iterator[Node]¶: an iterator over nodes after this one in document order, excluding its descendants

html: str¶: the HTML serialization of this node and its subtree

inner_html: str¶: the HTML serialization of this node’s children

inner_xml: str¶: the well-formed XML/XHTML serialization of this node’s children

insert_after(*nodes: Node) → None¶

Insert each node into this node’s parent right after this node, in order, with the same move-or-adopt rule as insert_before().

Parameters:

nodes – the nodes to insert.

Raises:

TypeError – if an argument is not a node, or is a Document.
ValueError – if this node has no parent, or a node is an ancestor of the insertion point (which would form a cycle).

insert_before(*nodes: Node) → None¶

Insert each node into this node’s parent right before this node, in order. A node already in a tree is moved; a node from another tree is adopted by copy.

Parameters:

nodes – the nodes to insert.

Raises:

TypeError – if an argument is not a node, or is a Document.
ValueError – if this node has no parent, or a node is an ancestor of the insertion point (which would form a cycle).

links() → list[Link]¶

Find every link in this node and its subtree. Beyond <a href>, this finds the URLs hidden in srcset/ping/archive lists, a <meta http-equiv=refresh> redirect, and CSS url()/@import in a style attribute or a <style> sheet.

Returns:: the Link records in document order.

main_content() → Element | None¶

Find the dominant content element under this node, the article body with navigation, sidebars, ads, comments and other boilerplate scored out. Scores the tree by content density (text length, comma count, tag and class/id weight, discounted by link density), the readability heuristic, in C.

Returns:: the winning content Element, or None when nothing reads as content.

main_text() → str¶

Render the main content under this node as layout-aware plain text, as to_text() renders main_content().

Returns:: the main content’s text, or an empty string when there is none.

matches(selector: str, /) → bool¶

Test this node against a CSS selector, evaluated with its own ancestors and siblings as context.

Parameters:: selector – the CSS selector.
Returns:: whether this node is an Element that matches.
Raises:: ValueError – the selector is not valid CSS; the message names the reason and the source offset.

next_sibling: Node | None¶: the following sibling node, or None

next_siblings: Iterator[Node]¶: an iterator over the following siblings in document order

parent: Element | Document | None¶: the parent Element or Document, or None for the document root

position: tuple[int, int] | None¶: the (source_line, source_col) of this element’s start tag, or None if unavailable

preceding: Iterator[Node]¶: an iterator over nodes before this one in document order, nearest first, excluding its ancestors

previous_sibling: Node | None¶: the preceding sibling node, or None

previous_siblings: Iterator[Node]¶: an iterator over the preceding siblings, nearest first

prune(selector: str, /) → Node¶

Keep only the descendants matching a CSS selector, together with their ancestors up to this node and the whole subtree under each match, and remove every other descendant in place. With no match the subtree is emptied. This trims a parsed document to the parts of interest after a normal WHATWG parse, the way BeautifulSoup’s SoupStrainer filters a document while parsing it.

Parameters:: selector – the CSS selector the kept descendants must match.
Returns:: this node.
Raises:: ValueError – the selector is not valid CSS; the message names the reason and the source offset.

re(pattern: str | Pattern[str], /, *, attr: str | None = None) → list[str]¶

Run a regex over this node’s text. Each match yields its one capturing group when the pattern has exactly one, otherwise the whole match.

Parameters:

pattern – a str or compiled re.Pattern to search for.
attr – search this attribute’s value instead of the node text; an absent attribute yields [].

Returns:

every match as a list of str.

re_first(pattern: str | Pattern[str], /, default: str | None = None, *, attr: str | None = None) → str | None¶

Run a regex over this node’s text and return the first match, with the same group rule as re().

Parameters:

pattern – a str or compiled re.Pattern to search for.
default – value returned when nothing matches.
attr – search this attribute’s value instead of the node text.

Returns:

the first match as a str, or default when nothing matches.

remove(selector: str, /) → Node¶

Drop every descendant Element matching the CSS selector, each with its whole subtree, and return this node. The bulk inverse of prune (which keeps the matches): the destructive counterpart of selectolax’s strip_tags and w3lib’s remove_tags_with_content, and of jQuery’s .remove().

Parameters:: selector – the CSS selector the dropped descendants must match.
Returns:: this node.
Raises:: ValueError – the selector is not valid CSS; the message names the reason and the source offset.

replace_with(*nodes: Node) → None¶

Put nodes where this node is, in order, and detach this node, which becomes a standalone root the caller still holds. With no nodes this just removes this node.

Parameters:

nodes – the nodes to put in this node’s place.

Raises:

TypeError – if an argument is not a node, or is a Document.
ValueError – if this node has no parent, or a node is an ancestor of this node (which would form a cycle).

resolve_links(base_url: str, /) → None¶

Rewrite every link in this node and its subtree to an absolute URL, in place, using stdlib urllib.parse.urljoin.

Parameters:: base_url – the base each relative URL is resolved against.

rewrite_links(replace: Callable[[str], str | None], /) → None¶

Rewrite every link in this node and its subtree in place.

Parameters:: replace – called with each URL; return a str to substitute it or None to leave it unchanged.

select(selector: str, /) → list[Element]¶

Find the descendant Elements matching a CSS selector. The grammar covers type, #id, .class, and attribute selectors, the four combinators, the structural pseudo-classes (including :nth-child(An+B of S)), the :is(), :where(), :has(), and :not() functional pseudo-classes, and the :scope, form/UI, :lang() and :dir() pseudo-classes a static tree can determine; live-state pseudo-classes (:hover, :focus, …) match nothing. :is() and :where() take a forgiving list, so a bad arm is dropped.

Parameters:: selector – the CSS selector.
Returns:: the matching descendant Elements in document order.
Raises:: ValueError – the selector is not valid CSS; the message names the reason and the source offset.

select_one(selector: str, /) → Element | None¶

Find the first descendant Element matching a CSS selector.

Parameters:: selector – the CSS selector.
Returns:: the first matching Element, or None.
Raises:: ValueError – the selector is not valid CSS; the message names the reason and the source offset.

serialize(options: Html | None = None) → str¶

Serialize this node and its subtree to a str.

Parameters:: options – an Html configuration object (formatter, layout, attribute ordering, and meta-charset handling), or None for the defaults.
Returns:: the serialized markup.
Raises:: TypeError – if options is not an Html configuration object.

serialize_iter(options: Html | None = None) → Iterator[str]¶

Serialize this node and its subtree lazily, yielding the markup in bounded str chunks so a large document can stream to a socket or file without a full-size output string. ''.join(node.serialize_iter(options)) equals node.serialize(options) for every options the stream supports.

The tree must not be mutated while the iterator is live, the same rule as the other node iterators.

Parameters:

options – an Html configuration object, or None for the defaults. A Minify layout is rejected: minification needs the whole tree at once.

Returns:

an iterator of str chunks whose concatenation is the markup.

Raises:

TypeError – if options is not an Html configuration object.
ValueError – if options selects a Minify layout, which cannot stream.

source_col: int | None¶: the 0-based source column of this element’s start tag, or None if unavailable

source_line: int | None¶: the 1-based source line of this element’s start tag, or None if unavailable

source_location: SourceLocation | None¶: the SourceLocation (start-/end-tag and per-attribute spans) of this element, or None when the tree was not parsed with source_locations or the element has no source start tag

strings: Iterator[str]¶: an iterator over the text of every Text descendant

strip_tags(selector: str, /) → Node¶

Unwrap every descendant Element matching the CSS selector, replacing each match with its children in place while keeping that content, and return this node. The bulk form of unwrap, matching selectolax’s unwrap_tags, w3lib’s remove_tags, and jQuery’s .unwrap().

Parameters:: selector – the CSS selector the unwrapped descendants must match.
Returns:: this node.
Raises:: ValueError – the selector is not valid CSS; the message names the reason and the source offset.

stripped_strings: Iterator[str]¶: strings with surrounding whitespace removed and blank runs skipped

tables() → list[list[list[str]]]¶: Return every table in this node and its subtree, in document order, each as the list of rows that Element.rows() produces. A nested table appears as its own entry. The result is a list[list[list[str]]] of plain strings, with no pandas dependency; pass one table to pandas.DataFrame for a frame.

text: str¶: the concatenated character data of every Text descendant

to_annotated_text(annotation_rules: Mapping[str, Sequence[str]], options: PlainText | None = None) → tuple[str, list[tuple[int, int, str]]]¶

Render layout-aware text and, for every element matching a rule in annotation_rules, record a labeled span over its text. Spans inside table cells are not recorded.

Parameters:

annotation_rules – maps a selector (‘tag’, ‘tag#attr’, ‘tag#attr=value’, or ‘#attr’) to the list of labels to attach to each matching element.
options – a PlainText configuration object, or None for the defaults.

Returns:

a (text, spans) pair, where spans is a list of (start, end, label).

to_markdown(options: Markdown | None = None) → str¶

Render this node and its subtree as Markdown. The defaults emit opinionated GitHub-Flavored Markdown.

Parameters:: options – a Markdown configuration object, or None for the defaults. Its grouped knobs (headings, links, tables, …) cover the markdownify and html2text configuration surface.
Returns:: the Markdown rendering of this node’s subtree.

to_source() → str¶

Losslessly serialize this node and its subtree back to a str, re-emitting the verbatim source bytes of every element and text run the parse left untouched and reserializing only the parts a mutation changed.

On a tree parsed with source_locations=True and not otherwise read, the round trip reproduces the source byte for byte – author quoting, tag-name case, character-reference spelling, and insignificant whitespace intact – for input that parsed without implied elements or content reordering. After a mutation only the changed node’s markup is rewritten: an element whose attributes changed rebuilds its start tag, an edited text run re-escapes, and an inserted element serializes canonically, while every untouched sibling and subtree still copies its original span. Without source locations every element reserializes canonically, so the result matches serialize().

Returns:: the lossless HTML serialization.

to_text(options: PlainText | None = None) → str¶

Render this node and its subtree as layout-aware plain text: blocks separated by blank lines, lists indented under their bullets, and tables laid out as a column-aligned grid. The inscriptis role, in C.

Parameters:: options – a PlainText configuration object, or None for the defaults.
Returns:: the plain-text rendering of this node’s subtree.

unwrap() → Node¶

Replace this node with its children, the inverse of wrap().

Returns:: this node, detached.
Raises:: ValueError – if this node has no parent.

wrap(wrapper: Element, /) → Element¶

Put this node inside wrapper, in this node’s place.

Parameters:: wrapper – the element to wrap this node in.
Returns:: wrapper, now holding this node.
Raises:: TypeError – if wrapper is not an element.

wrap_siblings(wrapper: Element, /, *, until: Node | None = None) → Element¶

Wrap this node and the siblings that follow it in wrapper in one move; the bulk form of wrap() for a contiguous run. wrapper lands where this node was.

Parameters:

wrapper – the element to wrap the run in.
until – the last sibling to include (this node or a later one); None reaches to the last sibling.

Returns:

wrapper, now holding the run.

Raises:

TypeError – if wrapper is not an element, or until is not a node.
ValueError – if this node has no parent, or until is not this node or a following sibling.

Evaluate an XPath expression relative to this node. A node-set returns a list of Elements (with attribute/text values as str) in document order; absolute paths start at the document root.

Parameters:

expression – the XPath expression.
smart_strings – return each string result as a value that remembers the Element it came from.
extensions – maps an (namespace, name) pair to a callable, registering a custom XPath function; the callable may return a scalar, an Element, or an iterable of Elements (a node-set that feeds later steps).
variables – values bound to the $name variables used in the expression; a str, int, float, or bool binds a scalar, and an Element or an iterable of Elements binds a node-set. A node from a different document raises ValueError.

Returns:

the result list, of Elements and str values in document order.

Raises:

TypeError – expression is not a str, a variable binding is of an unsupported type, or a value is used where the grammar requires a node-set.
ValueError – the expression is not valid XPath (with the offending token and its offset), nests past the depth limit, calls an unknown function or one with the wrong number of arguments, references an unbound variable or prefix, or binds a node from a different document.

Like xpath(), but stream the results instead of building a list.

Parameters:

expression – the XPath expression.
smart_strings – return each string result as a value that remembers the Element it came from.
extensions – maps an (namespace, name) pair to a custom XPath function.
variables – values bound to the $name variables in the expression.

Returns:

an iterator over the results in document order.

Raises:

TypeError – expression is not a str, a variable binding is of an unsupported type, or a value is used where a node-set is required.
ValueError – the expression is invalid XPath, nests too deeply, calls an unknown or wrong-arity function, or references an unbound variable or prefix.

Like xpath(), but return only the first result.

Parameters:

expression – the XPath expression.
smart_strings – return a string result as a value that remembers the Element it came from.
extensions – maps an (namespace, name) pair to a custom XPath function.
variables – values bound to the $name variables in the expression.

Returns:

the first result in document order, or None when there is none.

Raises:

TypeError – expression is not a str, a variable binding is of an unsupported type, or a value is used where a node-set is required.
ValueError – the expression is invalid XPath, nests too deeply, calls an unknown or wrong-arity function, or references an unbound variable or prefix.

class turbohtml.Element(tag: str, attrs: Mapping[str, str | list[str] | None] | None = None, children: Iterable[Node] | None = None)¶

An element node: a tag, a namespace, attributes, and child nodes.

Parameters:

tag – the tag name.
attrs – initial attributes; a list value sets a token-list attribute and None a valueless one.
children – initial child nodes, appended in order.

Raises:

TypeError – if tag or an attribute name is not a str, an attribute value is not a str, a list of str, or None, or a child is not a node.
ValueError – if the tag or an attribute name carries a character HTML forbids there, or if children are given for a void element.

add_class(name: str, /) → Element¶

Add name to this element’s class tokens when absent and return the element. Existing tokens keep their order; on a change the class value is rewritten with single-space separators.

Raises:

TypeError – if name is not a str.
ValueError – if name is empty or contains whitespace.

append(child: Node, /) → None¶

Add child as the last child of this element. A node already in a tree is moved; a node from another tree is adopted by copy.

Parameters:

child – the node to append.

Raises:

TypeError – if child is not a node, or is a Document.
ValueError – if child is an ancestor of this element (which would form a cycle).

assigned_elements(*, flatten: bool = False) → list[Element]¶

The elements assigned to this <slot> (assigned_nodes without the Text nodes), the DOM assignedElements.

Parameters:: flatten – return the flattened assignment instead of the direct one.
Returns:: the assigned elements as a list.
Raises:: TypeError – if the element is not a <slot>.

assigned_nodes(*, flatten: bool = False) → list[Node]¶

The nodes assigned to this <slot>, in tree order (the DOM assignedNodes).

A slot is assigned the host’s direct child nodes whose slot name matches its own name attribute (the default slot has the empty name). With flatten, empty slots fall back to their own children and nested shadow slots are expanded.

Parameters:: flatten – return the flattened assignment instead of the direct one.
Returns:: the assigned nodes as a list.
Raises:: TypeError – if the element is not a <slot>.

attach_shadow(mode: str = 'open') → ShadowRoot¶

Attach a shadow root to this element and return it, the DOM attachShadow.

The shadow root is a document-fragment-like container held off the light tree, so it never appears among this element’s children or in its serialization; build its content with ShadowRoot.set_inner_html or append.

Parameters:: mode – ‘open’ (Element.shadow_root exposes the root) or ‘closed’ (it reads None, so only this returned reference reaches the shadow tree).
Returns:: the new ShadowRoot.
Raises:: ValueError – if mode is not ‘open’ or ‘closed’, or the element already has a shadow root.

attr(name: str, /, default: str | None = None) → str | None¶

Read one attribute as a single str. The raw value is returned, so a token-list attribute like class reads back as “a b c” rather than a list, and a valueless attribute reads back as the empty string.

Parameters:

name – the attribute name.
default – value returned when the attribute is absent.

Returns:

the attribute value, or default when it is absent.

attrs: MutableMapping[str, str | list[str] | None]¶: the live mutable attribute mapping; token-list attributes (class, rel, …) map to a list[str], a valueless attribute maps to None

checked: bool¶: whether a checkbox or radio input is checked. Assigning requires a checkbox or radio; setting a radio to True clears the other same-name radios in the owning form (or document), the radio-group exclusivity rule.

clear() → None¶: Detach every child of this element, leaving it empty.

css_path() → str¶: Return a CSS selector that uniquely locates this element from the document root, the way browser devtools “copy selector” does. The path is anchored at the nearest ancestor (or the element itself) carrying a document-unique id (“#main > …”), otherwise it descends from the root with positional :nth-of-type() steps (“html > body > div > p:nth-of-type(3)”). Feeding the result back to select() on the document returns exactly this element.

extend(children: Iterable[Node], /) → None¶

Append every node from the iterable in order, each one moved or adopted like append().

Parameters:

children – the nodes to append.

Raises:

TypeError – if children is not iterable, or a member is not a node or is a Document.
ValueError – if a member is an ancestor of this element (which would form a cycle).

field_value: str | list[str] | None¶: the form control’s value, with form semantics. Reading returns the value attribute (defaulting to “on” for a checkbox/radio), a textarea’s text, an option’s value (its stripped text when it has no value attribute), or the selected option value(s) of a select (a list[str] when it is multiple, None when nothing is selected); non-controls read None. Assigning a str writes the value (selecting the matching option of a select), a list[str] selects a multiple select, and None clears it. The checked state lives in Element.checked, not here.

form_data() → list[tuple[str, str]]¶

Collect this form’s successful controls, following the WHATWG form-submission entry-list rules. Controls without a non-empty name, disabled controls (their own disabled or a disabling ancestor fieldset), buttons, and file/submit/reset/image inputs are skipped; a checkbox or radio contributes only when checked, a select one pair per selected non-disabled option (the default first option only when its display size is 1). Controls inside a template’s contents have no form owner and are excluded. Controls are matched by containment in the form.

Returns:: the (name, value) pairs in document order.

has_class(name: str, /) → bool¶: Return whether name is one of this element’s space-separated class tokens.

insert(index: int, child: Node, /) → None¶

Insert child among this element’s children, counted and clamped like list.insert.

Parameters:

index – position among the existing children.
child – the node to insert.

Raises:

TypeError – if index is not an int, or child is not a node or is a Document.
ValueError – if child is an ancestor of this element (which would form a cycle).

insert_adjacent_html(position: str, html: str, /) → None¶

Parse html as a fragment and insert it relative to this element at position, one of ‘beforebegin’, ‘afterbegin’, ‘beforeend’, or ‘afterend’ (the DOM insertAdjacentHTML, matched case-insensitively). ‘beforebegin’ and ‘afterend’ place the nodes among this element’s siblings, so they require an element parent; ‘afterbegin’ and ‘beforeend’ add them as the first or last children. The fragment parses in the context of the element that will hold it.

Raises:

TypeError – if position or html is not a str.
ValueError – if position is not one of the four keywords, or a sibling-relative position is used on a node without an element parent.

namespace: Namespace¶: the element’s Namespace (HTML, SVG, or MATHML)

normalize() → None¶: Merge each run of adjacent Text descendants into one node and drop empty Text nodes, throughout this element’s subtree.

records() → list[dict[str, str]]¶: Return the table’s data rows as a list of dicts, keyed by the first row (the header, typically the thead row) over each later row, with rowspan and colspan resolved as in rows(). A table with no rows or only a header yields an empty list; a duplicated header keeps the rightmost column’s value. Pass the result to pandas.DataFrame for a frame. Raises TypeError on a non-table element.

remove_class(name: str, /) → Element¶

Remove every occurrence of name from this element’s class tokens and return the element. Removing the last token leaves an empty class attribute.

Raises:

TypeError – if name is not a str.
ValueError – if name is empty or contains whitespace.

rows() → list[list[str]]¶: Return the table’s cells as a list of rows, each a list[str], with rowspan and colspan resolved by filling every spanned slot with a copy of the cell text. Rows are padded to a rectangular width; a nested table’s rows belong to that table, not this one. Cell text is the cell’s text content with surrounding whitespace stripped. Raises TypeError on a non-table element.

set_inner_html(html: str, /) → None¶

Replace this element’s children with the nodes parsed from html, a fragment parsed in this element’s own context (the DOM innerHTML= setter). The string is run through the same HTML parser as parse(), so malformed markup is repaired the same way.

Raises:: TypeError – if html is not a str.

set_text(text: str, /) → None¶

Replace this element’s children with a single Text node holding text verbatim (the DOM textContent= setter). text is never parsed, so any markup in it is escaped on serialization. The Element.text= setter is equivalent.

Raises:: TypeError – if text is not a str.

shadow_root: ShadowRoot | None¶: this element’s open shadow root, or None when it has none or the root is closed

tag: str¶: the lowercased tag name

text: str¶: the element’s text; assigning replaces all children with a single Text node

toggle_class(name: str, /) → Element¶

Remove name from this element’s class tokens when present, add it when absent, and return the element.

Raises:

TypeError – if name is not a str.
ValueError – if name is empty or contains whitespace.

wrap_children(wrapper: Element, /) → Element¶

Move every child of this element into wrapper, make wrapper the sole child, and return it. The bulk form of wrap() for a container’s whole content; an empty element gains an empty wrapper.

Parameters:: wrapper – the element to move the children into.
Returns:: wrapper, now holding the moved children.
Raises:: TypeError – if wrapper is not an element.

xpath_path() → str¶: Return the positional XPath that locates this element from the document root, like lxml’s getroottree().getpath(). Each step is the tag name with a 1-based [n] index among same-name siblings when more than one exists (“/html/body/div[2]/p[3]”). Feeding the result back to xpath() on the document returns exactly this element.

class turbohtml.Namespace(*values)¶

The XML namespace an Element belongs to, as reported by Element.namespace.

HTML is the ordinary HTML namespace; SVG is inline <svg> content; MATHML is inline <math> content. Each member’s value is the namespace short name (“html”, “svg”, “math”).

An element can host a shadow tree with Element.attach_shadow(), returning a ShadowRoot – a document-fragment-like root held off the light tree. <slot> elements inside it pull in the host’s children by name; Element.assigned_nodes(), Element.assigned_elements(), Node.assigned_slot, and Node.flattened_children read the assignment and the composed tree. See Use the Shadow DOM for recipes and Shadow DOM for the model.

class turbohtml.ShadowRoot¶

A shadow root: the document-fragment-like root of an element’s shadow tree, created by Element.attach_shadow. It is held off the light tree, so it never appears among the host’s children or in its serialization.

append(child: Node, /) → None¶: Add child as the last node of the shadow tree, moving a node from this tree or adopting one from another by copy, like Element.append.

clonable: bool¶: whether the shadow root is clonable, from a declarative shadow root’s shadowrootclonable attribute (always False otherwise)

delegates_focus: bool¶: whether the shadow root delegates focus, from a declarative shadow root’s shadowrootdelegatesfocus attribute (always False otherwise)

host: Element¶: the Element this shadow root is attached to

mode: str¶: the shadow root’s mode: ‘open’ or ‘closed’

set_inner_html(html: str, /) → None¶

Replace the shadow tree’s content by parsing html as a fragment, the way declarative shadow DOM populates a shadow root.

Parameters:: html – the markup to parse and install as the shadow content.
Raises:: TypeError – if html is not a str.

class turbohtml.Text(data: str)¶

A run of character data.

Parameters:: data – the text content.

data: str¶: the character data of this node

class turbohtml.Comment(data: str)¶

An HTML comment.

Parameters:: data – the comment text, without the <!– –> delimiters.

data: str¶: the character data of this node

class turbohtml.CData(data: str)¶

A CDATA section.

Parameters:: data – the section text, without the <![CDATA[ ]]> delimiters.

data: str¶: the character data of this node

class turbohtml.ProcessingInstruction(target: str, data: str)¶

A processing instruction.

Parameters:

target – the instruction target (the name right after <?).
data – the instruction data (everything after the target).

data: str¶: the instruction data (everything after the target)

target: str¶: the instruction target (the name right after <?)

class turbohtml.Doctype¶

A document type declaration.

name: str¶: the document type name

public_id: str | None¶: the public identifier, or None when the doctype has none

system_id: str | None¶: the system identifier, or None when the doctype has none

A Range marks a run of the tree between two boundary points (a (container, offset) pair each) and copies, moves, deletes, or wraps everything between them, following the DOM Living Standard. StaticRange is its immutable snapshot. See Ranges and boundary points for the boundary-point model and Work with ranges of the tree for recipes.

class turbohtml.Range(container: Node, offset: int = 0)¶

A live DOM Range between two boundary points.

Parameters:

container – the node both boundaries start collapsed in.
offset – the shared start offset (default 0).

clone_contents() → Node¶: Copy the range’s content into a new fragment.

clone_range() → Range¶: Return a new Range with the same boundary points.

collapse(to_start: bool = False, /) → None¶: Collapse the range onto one of its boundaries.

collapsed: bool¶: whether the two boundary points coincide

common_ancestor_container: Node¶: the deepest node containing both boundary points

compare_boundary_points(how: int, source_range: Range, /) → int¶: Compare a boundary of this range with one of source_range; returns -1, 0, or 1.

compare_point(node: Node, offset: int, /) → int¶: Where (node, offset) lies relative to the range: -1, 0, or 1.

delete_contents() → None¶: Remove the range’s content from the tree.

end_container: Node¶: the node the range ends in

end_offset: int¶: the offset of the end boundary within its container

extract_contents() → Node¶: Move the range’s content into a new fragment.

insert_node(node: Node, /) → None¶: Insert node at the start of the range.

intersects_node(node: Node, /) → bool¶: Whether node overlaps the range.

is_point_in_range(node: Node, offset: int, /) → bool¶: Whether (node, offset) is within the range.

select_node(node: Node, /) → None¶: Set the range to select node itself.

select_node_contents(node: Node, /) → None¶: Set the range to select node’s children.

set_end(container: Node, offset: int, /) → None¶: Set the end boundary to (container, offset).

set_end_after(node: Node, /) → None¶: Set the end boundary just after node.

set_end_before(node: Node, /) → None¶: Set the end boundary just before node.

set_start(container: Node, offset: int, /) → None¶: Set the start boundary to (container, offset).

set_start_after(node: Node, /) → None¶: Set the start boundary just after node.

set_start_before(node: Node, /) → None¶: Set the start boundary just before node.

start_container: Node¶: the node the range starts in

start_offset: int¶: the offset of the start boundary within its container

surround_contents(new_parent: Node, /) → Node¶: Wrap the range’s content in new_parent.

class turbohtml.StaticRange(start_container: Node, start_offset: int, end_container: Node, end_offset: int)¶

An immutable snapshot of two boundary points.

Parameters:

start_container – the node the range starts in.
start_offset – the start offset.
end_container – the node the range ends in.
end_offset – the end offset.

collapsed: bool¶: whether the two boundary points coincide

end_container: Node¶: the node the range ends in

end_offset: int¶: the offset of the end boundary within its container

start_container: Node¶: the node the range starts in

start_offset: int¶: the offset of the start boundary within its container

A MutationObserver records changes made to a subtree through the mutation API and hands them back as MutationRecord values. Delivery is synchronous: pull the batch with take_records(), or push it to a callback with deliver(). See Mutating the tree for the synchronous model and Observe tree mutations for recipes.

class turbohtml.MutationObserver(callback: Callable[[list[MutationRecord], MutationObserver], object] | None = None)¶

A synchronous watcher for tree mutations, modeled on the DOM MutationObserver.

Register a target with observe(), then read the recorded changes with take_records(), or call deliver() to hand them to callback synchronously. Unlike the DOM, delivery never happens on its own – turbohtml has no event loop – so a callback fires only when you drain the queue.

Parameters:: callback – called as callback(records, observer) by deliver(); optional when you only pull records with take_records().

deliver() → list[MutationRecord]¶

Drain the queued records and pass them to the callback synchronously.

The DOM schedules this on a microtask; with no event loop turbohtml runs it when you call deliver(). Does nothing when the queue is empty or no callback was given. Returns the records that were delivered.

disconnect() → None¶: Stop observing every target and discard the queued records.

observe(target: Node, *, child_list: bool = False, attributes: bool = False, character_data: bool = False, subtree: bool = False, attribute_old_value: bool = False, character_data_old_value: bool = False, attribute_filter: Sequence[str] | None = None) → None¶

Watch target for mutations, recording each matching change until disconnect().

Parameters:

target – the node to observe; observing a second target adds to the set, re-observing one replaces its options. Every target must share one tree.
child_list – record additions and removals among target’s children.
attributes – record attribute changes; implied by attribute_old_value or attribute_filter.
character_data – record text changes on a Text/Comment/CData node; implied by character_data_old_value.
subtree – extend the watch to target’s whole subtree, not just its children.
attribute_old_value – record the previous value on each attribute record.
character_data_old_value – record the previous text on each character-data record.
attribute_filter – only record changes to these attribute names.

Raises:

TypeError – target is not a node, or the options set nothing to observe.
ValueError – target belongs to a different tree than an earlier one.

take_records() → list[MutationRecord]¶: Return the queued MutationRecords and empty the queue.

class turbohtml.MutationRecord(type: str, target: Node, added_nodes: tuple[Node, ...], removed_nodes: tuple[Node, ...], previous_sibling: Node | None, next_sibling: Node | None, attribute_name: str | None, old_value: str | None)[source]¶

One change delivered to an observer, mirroring the DOM MutationRecord.

Parameters:

type (str)
target (Node)
added_nodes (tuple[Node, ...])
removed_nodes (tuple[Node, ...])
previous_sibling (Node | None)
next_sibling (Node | None)
attribute_name (str | None)
old_value (str | None)

type: str¶: "childList", "attributes", or "characterData".

target: Node¶: the node the change applies to: the parent for a childList change, else the changed node.

added_nodes: tuple[Node, ...]¶: the nodes added, for a childList change (else empty).

removed_nodes: tuple[Node, ...]¶: the nodes removed, for a childList change (else empty).

previous_sibling: Node | None¶: the sibling before the added or removed nodes, or None.

next_sibling: Node | None¶: the sibling after the added or removed nodes, or None.

attribute_name: str | None¶: the changed attribute’s name, for an attributes change (else None).

old_value: str | None¶: the previous value, when the registration asked for it (else None).

Node.links() yields one Link per link it finds.

class turbohtml.Link(element: Element, attribute: str | None, url: str)[source]¶

One link found in a document.

Parameters:

element (Element) – the element the link was found on.
attribute (str | None) – the attribute carrying the link, or None for a <style> sheet’s text.
url (str) – the link exactly as it appears in the source, before any resolution.

element: Element¶: the element the link was found on.

attribute: str | None¶: the attribute carrying the link, or None for a <style> sheet’s text.

url: str¶: the link exactly as it appears in the source, before any resolution.

Node.article() returns an Article record: the scored content body and the page metadata harvested beside it.

The dominant content of a document and the metadata harvested beside it.

element is the scored content body (None when nothing reads as content) and text its layout-aware plain text (empty then). title, byline, date, description, lang, canonical (the <link rel=canonical> URL or og:url), site_name (og:site_name or <meta name=application-name>) and image (the og:image or twitter:image lead image) are single-valued page metadata, each whitespace-normalized or None when the source is absent. tags collects every <meta name=keywords> value (comma-split) and article:tag in document order, an empty tuple when none appear. This makes article() a one-call replacement for trafilatura and newspaper3k.

Parameters:

element (Element | None)
text (str)
title (str | None)
byline (str | None)
date (str | None)
description (str | None)
lang (str | None)
canonical (str | None)
site_name (str | None)
tags (tuple[str, ...])
image (str | None)

element: Element | None¶: Alias for field number 0

text: str¶: Alias for field number 1

title: str | None¶: Alias for field number 2

byline: str | None¶: Alias for field number 3

date: str | None¶: Alias for field number 4

description: str | None¶: Alias for field number 5

lang: str | None¶: Alias for field number 6

canonical: str | None¶: Alias for field number 7

site_name: str | None¶: Alias for field number 8

tags: tuple[str, ...]¶: Alias for field number 9

image: str | None¶: Alias for field number 10

When a tree is parsed with source_locations=True, Node.source_location returns a SourceLocation record built from SourceSpan values – the start-tag, end-tag, and per-attribute spans parse5 exposes as sourceCodeLocationInfo.

class turbohtml.SourceLocation(start_tag: SourceSpan, end_tag: SourceSpan | None, attrs: dict[str, SourceSpan])[source]¶

Where a parsed element sat in the source.

start_tag spans the <tag ...>; end_tag spans the </tag> or is None when the source never closed the element (a void, self-closed, or implicitly/EOF-closed one). attrs maps each attribute’s name to the span covering its name="value" (the name alone for a valueless attribute), keyed by the same lowercased name attrs uses, first occurrence winning on a duplicate.

Parameters:

start_tag (SourceSpan)
end_tag (SourceSpan | None)
attrs (dict[str, SourceSpan])

start_tag: SourceSpan¶: the span of the element’s start tag.

end_tag: SourceSpan | None¶: the span of the element’s end tag, or None when the source did not close it.

attrs: dict[str, SourceSpan]¶: each attribute name mapped to the span of its whole name="value".

class turbohtml.SourceSpan(start_line: int, start_col: int, start_offset: int, end_line: int, end_col: int, end_offset: int)[source]¶

Where one construct sat in the source: its start and end line, column, and code-point offset.

Lines are 1-based, columns 0-based, and offsets are code-point indices into the newline-normalized source (a \\r\\n counts once), the same convention as source_line and source_col. The half-open [start_offset, end_offset) slices the construct out of the source.

Parameters:

start_line (int)
start_col (int)
start_offset (int)
end_line (int)
end_col (int)
end_offset (int)

start_line: int¶: the 1-based line the construct starts on.

start_col: int¶: the 0-based column the construct starts at.

start_offset: int¶: the code-point offset the construct starts at.

end_line: int¶: the 1-based line just past the construct’s end.

end_col: int¶: the 0-based column just past the construct’s end.

end_offset: int¶: the code-point offset just past the construct’s end.

Traversal objects¶

The DOM Living Standard traversal objects walk a subtree under a NodeFilter bitmask and callback. A TreeWalker is a movable cursor; a NodeIterator is a flat forward/backward view. See Walk a tree with a cursor and filter for recipes and Traversal objects for the reject/skip semantics.

class turbohtml.NodeFilter[source]¶

The whatToShow bits and filter verdicts a TreeWalker or NodeIterator reads.

The SHOW_* constants are the what_to_show bitmask – OR them to consider several node types, or use SHOW_ALL. A filter callback returns one of the FILTER_* verdicts: FILTER_ACCEPT yields the node, FILTER_REJECT skips it and (in a TreeWalker) its whole subtree, and FILTER_SKIP skips only the node. A NodeIterator has no subtree to skip, so it treats reject and skip alike.

SHOW_ALL: Final = 4294967295¶

SHOW_ELEMENT: Final = 1¶

SHOW_ATTRIBUTE: Final = 2¶

SHOW_TEXT: Final = 4¶

SHOW_CDATA_SECTION: Final = 8¶

SHOW_ENTITY_REFERENCE: Final = 16¶

SHOW_ENTITY: Final = 32¶

SHOW_PROCESSING_INSTRUCTION: Final = 64¶

SHOW_COMMENT: Final = 128¶

SHOW_DOCUMENT: Final = 256¶

SHOW_DOCUMENT_TYPE: Final = 512¶

SHOW_DOCUMENT_FRAGMENT: Final = 1024¶

SHOW_NOTATION: Final = 2048¶

FILTER_ACCEPT: Final = 1¶

FILTER_REJECT: Final = 2¶

FILTER_SKIP: Final = 3¶

class turbohtml.TreeWalker(root: Node, what_to_show: int = ..., filter: Callable[[Node], int] | None = ...)¶

A movable cursor over the nodes of root’s subtree the filter accepts.

The DOM Living Standard TreeWalker. what_to_show is a NodeFilter bitmask of node types to consider; filter is a callable taking a Node and returning NodeFilter.FILTER_ACCEPT, FILTER_REJECT, or FILTER_SKIP. REJECT skips the node and its whole subtree; SKIP skips only the node. current_node starts at root and each move returns the new node, or None when there is none in that direction (leaving current_node unchanged).

Parameters:

root – the node whose subtree the walk is confined to.
what_to_show – the whatToShow bitmask; SHOW_ALL by default.
filter – a Node -> int callback, or None to test only what_to_show.

Raises:

TypeError – root is not a Node, or filter is neither callable nor None.
ValueError – a filter re-enters the walker, or current_node is set to a node from another tree.

current_node: Node¶: the node the cursor rests on; assignable to any node in the same tree

filter: Callable[[Node], int] | None¶: the NodeFilter callback, or None

first_child() → Node | None¶: Move to the first accepted child of the current node and return it, or None.

last_child() → Node | None¶: Move to the last accepted child of the current node and return it, or None.

next_node() → Node | None¶: Move to the next accepted node in document order and return it, or None.

next_sibling() → Node | None¶: Move to the next accepted sibling of the current node and return it, or None.

parent_node() → Node | None¶: Move to the nearest accepted ancestor within root and return it, or None.

previous_node() → Node | None¶: Move to the previous accepted node in document order and return it, or None.

previous_sibling() → Node | None¶: Move to the previous accepted sibling of the current node and return it, or None.

root: Node¶: the node the walk is rooted at (read-only)

what_to_show: int¶: the whatToShow bitmask of node types the walk considers

class turbohtml.NodeIterator(root: Node, what_to_show: int = ..., filter: Callable[[Node], int] | None = ...)¶

A flat forward/backward view of the nodes of root’s subtree the filter accepts.

The DOM Living Standard NodeIterator. what_to_show and filter work as for TreeWalker, but because the view is flat there is no subtree to skip: FILTER_REJECT and FILTER_SKIP behave identically. next_node and previous_node walk the accepted nodes in document order and return None past an end; iterating the object with a for loop yields the same forward sequence, stopping at the end.

Parameters:

root – the node whose subtree the iteration is confined to.
what_to_show – the whatToShow bitmask; SHOW_ALL by default.
filter – a Node -> int callback, or None to test only what_to_show.

Raises:

TypeError – root is not a Node, or filter is neither callable nor None.
ValueError – a filter re-enters the iterator.

filter: Callable[[Node], int] | None¶: the NodeFilter callback, or None

next_node() → Node | None¶: Return the next accepted node in document order, or None past the end.

pointer_before_reference_node: bool¶: whether the iterator sits just before (True) or after (False) reference_node

previous_node() → Node | None¶: Return the previous accepted node in document order, or None past the start.

reference_node: Node¶: the node the iterator last returned or started at

root: Node¶: the node the iteration is rooted at (read-only)

what_to_show: int¶: the whatToShow bitmask of node types the iteration considers