all2md.parsers.asciidoc

AsciiDoc to AST converter.

This module provides conversion from AsciiDoc documents to AST representation using a custom parser implementation. It enables bidirectional transformation by parsing AsciiDoc into the same AST structure used for other formats.

class all2md.parsers.asciidoc.TokenType

Bases: Enum

Token types for AsciiDoc lexer.

HEADING = 1
PARAGRAPH = 2
CODE_BLOCK_DELIMITER = 3
QUOTE_BLOCK_DELIMITER = 4
LITERAL_BLOCK_DELIMITER = 5
SIDEBAR_BLOCK_DELIMITER = 6
EXAMPLE_BLOCK_DELIMITER = 7
TABLE_DELIMITER = 8
THEMATIC_BREAK = 9
UNORDERED_LIST = 10
ORDERED_LIST = 11
DESCRIPTION_TERM = 12
CHECKLIST_ITEM = 13
ATTRIBUTE = 14
BLOCK_ATTRIBUTE = 15
ANCHOR = 16
COMMENT = 17
BLANK_LINE = 18
TEXT_LINE = 19
EOF = 20
class all2md.parsers.asciidoc.Token

Bases: object

Represents a token from the lexer.

Parameters:

  • type (TokenType) – Type of the token

  • content (str) – Token content/value

  • line_num (int) – Line number in source

  • indent (int) – Indentation level

  • metadata (dict) – Additional token metadata

type: TokenType
content: str
line_num: int
indent: int = 0
metadata: dict[str, Any] | None = None
__init__(type: TokenType, content: str, line_num: int, indent: int = 0, metadata: dict[str, Any] | None = None) → None
class all2md.parsers.asciidoc.AsciiDocLexer

Bases: object

Tokenizer for AsciiDoc content.

This lexer performs line-by-line tokenization of AsciiDoc content, identifying block delimiters, list markers, attributes, and text.

Parameters:

content (str) – AsciiDoc content to tokenize

Initialize the lexer with content.

__init__(content: str)

Initialize the lexer with content.

tokens: list[Token]
tokenize() → list[Token]

Tokenize the content into a list of tokens.

Returns:

List of tokens

Return type:

list[Token]

class all2md.parsers.asciidoc.AsciiDocParser

Bases: BaseParser

Convert AsciiDoc to AST representation.

This parser implements a custom AsciiDoc parser that converts AsciiDoc documents into the all2md AST format. It uses a two-stage process: lexing (tokenization) and parsing.

Supported Features

  • Headings (= through ======)

  • Paragraphs with hard line breaks (trailing `` +``)

  • Inline formatting: - Bold: *text* (constrained), **text** (unconstrained) - Italic: _text_ (constrained), __text__ (unconstrained) - Monospace: `code` - Superscript: ^text^ - Subscript: ~text~ - Escape sequences: \*, _, \{, \+, \#, \!, \:, etc.

  • Lists with nesting: - Unordered (* through *****) - Ordered (. through …..) - Checklists (* [x] or * [ ]) - Description lists (term:: or term;)

  • Code blocks (—-) with language support via [source,lang]

  • Literal blocks (….) for preformatted text

  • Block quotes (____)

  • Sidebar blocks (****) rendered as block quotes with role=’sidebar’

  • Example blocks (====) rendered as block quotes with role=’example’

  • Tables (|===) with: - Attribute-based header detection - Escaped pipes (\|) in cells - Basic cell formatting

  • Links: link:url[text] and auto-links (http://…)

  • Images: image::url[alt] and image:url[alt]

  • Cross-references: <<id>> and <<id,text>>

  • Attribute references in braces

  • Block attributes: [#id], [.role], [source,python], [options=”header”]

  • Anchors: [[anchor-id]]

  • Thematic breaks (‘’’, —, ***, ___, or 3+ of same char)

  • Passthrough: ++text++, +text+, pass:[text]

  • Document attributes (:name: value, :name!: to unset, multi-line with +)

  • Admonitions: [NOTE], [TIP], [IMPORTANT], [WARNING], [CAUTION]

  • Comments: // single-line comments (preserved as Comment nodes unless strip_comments=True)

Limitations

  • No support for: includes, conditionals, complex macros

  • Tables: no multi-line cells or nested tables (cell spanning is supported)

  • Some advanced inline formatting edge cases

param options:

Parser configuration options

type options:

AsciiDocOptions or None, default = None

param progress_callback:

Optional callback for progress updates

type progress_callback:

ProgressCallback or None, default = None

Examples

Basic parsing:

>>> parser = AsciiDocParser()
>>> doc = parser.parse("= Title\n\nThis is *bold*.")

With options:

>>> options = AsciiDocOptions(
...     support_unconstrained_formatting=True,
...     table_header_detection="attribute-based"
... )
>>> parser = AsciiDocParser(options)
>>> doc = parser.parse(asciidoc_text)

Initialize the AsciiDoc parser.

__init__(options: AsciiDocOptions | None = None, progress_callback: Callable[[ProgressEvent], None] | None = None)

Initialize the AsciiDoc parser.

options: AsciiDocOptions
tokens: list[Token]
attributes: dict[str, str]
pending_block_attrs: dict[str, Any]
parse(input_data: str | Path | IO[bytes] | bytes) → Document

Parse AsciiDoc input into AST Document.

Parameters:

input_data (str, Path, IO[bytes], or bytes) – AsciiDoc input to parse. Can be: - File path (str or Path) - File-like object in binary mode - Raw AsciiDoc bytes - AsciiDoc string

Returns:

AST document node

Return type:

Document

Raises:

ParsingError – If parsing fails

extract_metadata(document: Any) → DocumentMetadata

Extract metadata from AsciiDoc document.

Parameters:

document (Any) – Document content or AST

Returns:

Extracted metadata

Return type:

DocumentMetadata