all2md.parsers.html

HTML to AST converter.

This module provides conversion from HTML documents to AST representation. It replaces direct markdown string generation with structured AST building, enabling multiple rendering strategies and improved testability.

class all2md.parsers.html.HtmlToAstConverter

Bases: BaseParser

Convert HTML to AST representation.

This converter parses HTML using BeautifulSoup and builds an AST that can be rendered to various markdown flavors.

Parameters:: options (HtmlOptions or None, default = None) – Conversion options

Initialize the HTML parser with options and progress callback.

BLOCK_ELEMENTS = frozenset({'address', 'article', 'aside', 'blockquote', 'caption', 'center', 'dd', 'details', 'dialog', 'div', 'dl', 'dt', 'en-note', 'fieldset', 'figcaption', 'figure', 'footer', 'form', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'header', 'hgroup', 'hr', 'legend', 'li', 'main', 'nav', 'ol', 'p', 'pre', 'section', 'summary', 'table', 'tbody', 'td', 'tfoot', 'th', 'thead', 'tr', 'ul'})

INLINE_ELEMENTS = frozenset({'a', 'abbr', 'b', 'bdi', 'bdo', 'br', 'cite', 'code', 'data', 'del', 'dfn', 'em', 'i', 'img', 'ins', 'kbd', 'mark', 'q', 's', 'samp', 'small', 'span', 'strong', 'sub', 'sup', 'svg', 'time', 'u', 'var', 'wbr'})

__init__(options: HtmlOptions | None = None, progress_callback: Callable[[ProgressEvent], None] | None = None): Initialize the HTML parser with options and progress callback.

parse(input_data: str | Path | IO[bytes] | bytes) → Document

Parse HTML document into an AST.

Parameters:

input_data (str, Path, IO[bytes], or bytes) – The input HTML document to parse. Can be: - File path (str or Path) - File-like object in binary mode - Raw HTML bytes - HTML string content

Returns:

AST Document node representing the parsed HTML structure

Return type:

Document

Raises:

ParsingError – If parsing fails due to invalid HTML or corruption
FileAccessError – If input file cannot be accessed
MalformedFileError – If input data is malformed
ValidationError – If input type is not supported

convert_to_ast(html_content: str) → Document

Convert HTML string to AST Document.

Parameters:: html_content (str) – HTML content to convert
Returns:: AST document node
Return type:: Document

extract_metadata(document: Any) → DocumentMetadata

Extract metadata from HTML document.

This method extracts metadata from the parsed HTML document, including title, author, description, keywords, and other standard metadata fields from HTML head section meta tags and Open Graph properties.

Parameters:: document (BeautifulSoup) – Parsed HTML document (BeautifulSoup object)
Returns:: Extracted metadata including title, author, subject, keywords, language, creator, category, and custom fields. Returns empty DocumentMetadata if no metadata is available.
Return type:: DocumentMetadata

Notes

This method extracts metadata from:

<title> tag in HTML head
<meta> tags with various name/property attributes
<link> tags with rel attributes
Open Graph (og:*) and Twitter Card (twitter:*) meta tags
Dublin Core (dc.*) meta tags
Article meta tags (article:*)

The method maps common meta tag names to standardized DocumentMetadata fields and stores unmapped tags in the custom dictionary.

Examples

>>> from bs4 import BeautifulSoup
>>> html = '<html><head><title>My Page</title><meta name="author" content="John Doe"></head></html>'
>>> soup = BeautifulSoup(html, "html.parser")
>>> converter = HtmlToAstConverter()
>>> metadata = converter.extract_metadata(soup)
>>> metadata.title
'My Page'
>>> metadata.author
'John Doe'