#############
 From parse5
#############

.. package-meta:: npm parse5 inikulin/parse5

`parse5 <https://github.com/inikulin/parse5>`_ is the reference JavaScript WHATWG parser -- the tree builder behind
jsdom, Angular, and much of the Node HTML ecosystem. It is unusual among the libraries here in being JavaScript rather
than Python, so this guide is a cross-language reference: it maps the parse5 API, and its ``sourceCodeLocationInfo``
source-location model in particular, onto turbohtml for teams moving an HTML pipeline from Node to Python.

Both build the same WHATWG tree from the same algorithm, so the port is mechanical. Where parse5 hands back a plain
object tree and leaves querying, mutation, and serialization to other packages (``@parse5/tools``, ``dom-serializer``,
your own walks), turbohtml keeps you inside one typed :class:`~turbohtml.Document` for all of it.

*********************
 turbohtml vs parse5
*********************

.. list-table::
    :header-rows: 1
    :widths: 16 42 42

    - - Dimension
      - turbohtml
      - parse5
    - - Language
      - Python, over a C engine
      - JavaScript (Node/browser)
    - - Scope
      - Parse, query, mutate, and serialize in one library
      - Parse and serialize; querying and mutation are your own walks or extra packages
    - - Source locations
      - :attr:`~turbohtml.Node.source_location` on each element under ``parse(source_locations=True)``
      - ``sourceCodeLocation`` on each node under ``{sourceCodeLocationInfo: true}``
    - - Querying
      - CSS :meth:`~turbohtml.Node.select`, XPath 1.0 :meth:`~turbohtml.Node.xpath`, the
        :meth:`~turbohtml.Node.find`/:meth:`~turbohtml.Node.find_all` grammar
      - none built in; walk ``childNodes`` or add a package
    - - Typing
      - Fully type annotated with bundled stubs
      - TypeScript types shipped with the package
    - - Performance
      - Native C engine straight into the native tree; see the table below
      - JavaScript over the same algorithm

Source-location parity
======================

parse5's ``sourceCodeLocationInfo`` option is the model turbohtml's ``source_locations`` matches. Both attach, to each
element, the span of its start tag, its end tag, and each attribute; both count a 1-based line, a 0-based column, and a
code-point offset; and both leave the end tag absent when the source never closed the element. The field names differ
but line up one for one:

.. list-table::
    :header-rows: 1
    :widths: 50 50

    - - parse5 ``sourceCodeLocation``
      - turbohtml :class:`~turbohtml.SourceLocation`
    - - ``loc.startTag`` (``{startLine, startCol, startOffset, endLine, endCol, endOffset}``)
      - ``location.start_tag`` (a :class:`~turbohtml.SourceSpan`)
    - - ``loc.endTag`` (``null`` when unclosed)
      - ``location.end_tag`` (``None`` when unclosed)
    - - ``loc.attrs[name]``
      - ``location.attrs[name]``
    - - ``span.startOffset`` / ``span.endOffset``
      - ``span.start_offset`` / ``span.end_offset``
    - - ``span.startLine`` / ``span.startCol``
      - ``span.start_line`` / ``span.start_col``

One difference: parse5 also fills a ``sourceCodeLocation`` on text, comment, and document-fragment nodes, and a
whole-element span (its ``startOffset``/``endOffset`` across children). turbohtml scopes
:attr:`~turbohtml.Node.source_location` to elements and reports ``None`` for other node types; a text run's own offsets
are not exposed. Read a node's :attr:`~turbohtml.Node.source_line`/:attr:`~turbohtml.Node.source_col` for the coarse
position that stays available on every parse.

Performance
===========

turbohtml records the same spans in its C engine, so parsing the same document with locations on runs 1.5x to 3.5x
faster than parse5 measured in-process (Node startup excluded):

.. bench-table::
    :file: bench/parse5.json

************************
 Declarative shadow DOM
************************

parse5 has no shadow-tree model: it builds a plain ``<template>`` element with a content fragment for a ``<template
shadowrootmode>`` and leaves the shadow-root attachment to its consumer (jsdom does it in its own layer; a custom
``treeAdapter`` would do it in yours). turbohtml attaches the shadow root in the parser itself and hands it back on the
host, so there is nothing extra to wire up:

.. testcode::

    from turbohtml import parse

    host = parse("<section id=h><template shadowrootmode=open><slot></slot></template><p>x</p></section>").find(id="h")
    print(host.shadow_root.mode)

.. testoutput::

    open

The switch maps to the ``allow_declarative_shadow_roots`` option, which follows the WHATWG per-document flag: on by
default for whole-document :func:`~turbohtml.parse` (a browser navigation), off for :func:`~turbohtml.parse_fragment`
(an ``innerHTML`` assignment). Read the attached tree through :attr:`~turbohtml.Element.shadow_root`,
:meth:`~turbohtml.Element.assigned_nodes`, and :attr:`~turbohtml.Node.flattened_children` -- the same API a scripted
:meth:`~turbohtml.Element.attach_shadow` produces.

**********************
 Custom tree adapters
**********************

parse5's ``treeAdapter`` option lets you retarget the parser at a tree of your own rather than its default object model,
the way jsdom plugs in its own node types. turbohtml's :func:`turbohtml.treebuild.parse_into` is the same idea: pass a
builder object and it drives the WHATWG tree builder straight into your representation, no :class:`~turbohtml.Document`
built on the way. The method set lines up with a parse5 adapter -- ``create_element`` receives a namespace URI and the
attributes as ``(name, value)`` pairs, ``append`` links a child under its parent -- so an adapter ports to a builder
method for method:

.. testcode::

    from turbohtml.treebuild import parse_into


    class Adapter:
        def create_document(self):
            return {"tag": "#document", "children": []}

        def create_doctype(self, name, public_id, system_id):
            return {"tag": "#doctype", "children": []}

        def create_element(self, name, namespace, attrs):
            return {"tag": name, "ns": namespace, "children": []}

        def create_text(self, data):
            return {"tag": "#text", "children": []}

        def create_comment(self, data):
            return {"tag": "#comment", "children": []}

        def create_pi(self, data):
            return {"tag": "#pi", "children": []}

        def append(self, parent, child):
            parent["children"].append(child)


    document = parse_into("<a href=/x>link</a>", Adapter())
    anchor = document["children"][0]["children"][1]["children"][0]
    print(anchor["tag"], anchor["ns"])

.. testoutput::

    a http://www.w3.org/1999/xhtml

Where parse5 folds ``<?...>`` into a comment, turbohtml routes it to a separate ``create_pi`` so you can keep it
distinct, and a ``<template>``'s content is appended straight under the template handle. See :doc:`/how-to/treebuild`
for the full builder recipe.

****************
 How to migrate
****************

.. code-block:: javascript

    // parse5
    import { parse } from "parse5";

    const document = parse(html, { sourceCodeLocationInfo: true });
    const el = document.childNodes[1]; // walk childNodes by hand
    const loc = el.sourceCodeLocation;
    html.slice(loc.startTag.startOffset, loc.startTag.endOffset);
    html.slice(loc.attrs.id.startOffset, loc.attrs.id.endOffset);

The turbohtml port parses the same way, then reads the span through :attr:`~turbohtml.Node.source_location` and slices
with plain offsets:

.. testcode::

    from turbohtml import parse

    source = '<html><body><div id="x">y</div></body></html>'
    div = parse(source, source_locations=True).find("div")
    location = div.source_location
    print(source[location.start_tag.start_offset : location.start_tag.end_offset])
    print(source[location.attrs["id"].start_offset : location.attrs["id"].end_offset])

.. testoutput::

    <div id="x">
    id="x"

**********************
 Gotchas and pitfalls
**********************

- **Opt in explicitly.** parse5's ``sourceCodeLocationInfo`` defaults off; so does turbohtml's ``source_locations``.
  Pass ``parse(html, source_locations=True)`` -- reading :attr:`~turbohtml.Node.source_location` on a tree parsed
  without it returns ``None``, never raises.
- **Only elements carry the record.** parse5 attaches ``sourceCodeLocation`` to text and comment nodes too; turbohtml
  reports ``None`` for non-element nodes. Port any code that read a text node's offsets to work from its surrounding
  elements.
- **Offsets are code points, not UTF-16 units.** parse5's offsets index a JavaScript string (UTF-16 code units), so an
  astral character counts as two; turbohtml's offsets are Python code points, so it counts as one. Slice each library's
  offsets against its own string type and the results agree.
- **Newlines are normalized.** As in parse5, a ``\r\n`` collapses to one line break before the counters advance, so line
  and offset numbers match the normalized source rather than the raw bytes.
- **No pluggable tree adapter.** parse5's ``treeAdapter`` lets you build a custom node shape; turbohtml always builds
  its own :class:`~turbohtml.Document`. To feed another toolchain, serialize with :attr:`~turbohtml.Node.html` and
  reparse there.
- **Declarative shadow is on for documents.** Unlike parse5, turbohtml attaches a shadow root for a ``<template
  shadowrootmode>`` during a whole-document :func:`~turbohtml.parse`, so such templates leave the light tree. Pass
  ``allow_declarative_shadow_roots=False`` to keep parse5's plain-template behavior, or opt a fragment in with
  ``parse_fragment(..., allow_declarative_shadow_roots=True)``.