Advanced Modules

These modules provide advanced functionality for working with document ASTs, search indexing, document comparison, and LLM integration.

AST (Abstract Syntax Tree)

The AST module defines the document structure used internally by all2md. All parsers produce AST documents, and all renderers consume them.

For user-facing AST documentation, see Working with the AST.

Core Classes

`all2md.ast.nodes.Document`	Root document node containing all other nodes.
`all2md.ast.nodes.Node`	Base class for all AST nodes.
`all2md.ast.builder.DocumentBuilder`	Helper for building complete documents.
`all2md.ast.visitors.NodeVisitor`	Abstract base class for AST node visitors.

Key Node Types

Block-level nodes:

`all2md.ast.nodes.Heading`	Heading node (h1-h6).
`all2md.ast.nodes.Paragraph`	Paragraph node containing inline content.
`all2md.ast.nodes.Table`	Table node with optional header and alignment.
`all2md.ast.nodes.CodeBlock`	Code block node with optional language specification.
`all2md.ast.nodes.List`	List node (ordered or unordered).
`all2md.ast.nodes.BlockQuote`	Block quote node containing other block elements.

Inline nodes:

`all2md.ast.nodes.Text`	Plain text node.
`all2md.ast.nodes.Link`	Link node.
`all2md.ast.nodes.Image`	Image node.
`all2md.ast.nodes.Code`	Inline code node.
`all2md.ast.nodes.Emphasis`	Emphasis (italic) node.
`all2md.ast.nodes.Strong`	Strong (bold) node.

AST Utilities

`all2md.ast.serialization`	JSON serialization and deserialization for AST nodes.
`all2md.ast.sections`	Core section utilities for document AST structures.
`all2md.ast.splitting`	Document splitting strategies.
`all2md.ast.transforms`	AST transformation and manipulation utilities.
`all2md.ast.utils`	Utility functions for working with AST nodes.

Search Module

Full-text and semantic search over converted documents:

`all2md.search.service`	High-level orchestration for indexing and querying documents.
`all2md.search.index`	Index abstractions and manifest utilities for search backends.
`all2md.search.chunking`	Chunk generation utilities for document indexing.
`all2md.search.types`	Shared data structures for the search subsystem.

Search Backends

`all2md.search.bm25`	BM25 keyword search index backed by rank-bm25.
`all2md.search.vector`	Vector search index built on FAISS and sentence-transformers.
`all2md.search.hybrid`	Utilities for combining multiple search backends.

Diff Module

Document comparison and diff rendering:

`all2md.diff.text_diff`	Simple text-based document comparison using difflib.
`all2md.diff.renderers`	Diff renderers for various output formats.

Quality Scoring

Three reads on conversion quality, answering three different questions.

all2md.confidence answers “can I trust this conversion?” It is reference-free, scoring from the sanity signals the parser observed, which is the only option for a document with no ground truth (a scanned PDF).

all2md.roundtrip answers “what did this conversion lose?” It manufactures a reference by converting a document to another format and parsing it straight back, then scores the structure that survived.

all2md.optimize answers “which of these settings is best?” — a distinct question, and it needs a distinct objective. The confidence score cannot be used to search: it saturates at 100 on anything not visibly broken, so across a sweep of converter options it is flat and there is no gradient to climb. The round-trip score cannot be used either, because re-parsing rendered output measures the renderer — a garbled table round-trips through Markdown perfectly. So the optimizer scores the parsed AST directly, on how much well-formed structure each setting recovers.

`all2md.confidence`	Conversion confidence reporting — a structured "quality card".
`all2md.roundtrip`	Round-trip fidelity scoring — how much structure survives a conversion.
`all2md.optimize`	Auto-tune converter options against a reference-free fidelity objective.

MCP Server

Model Context Protocol server for LLM integration:

`all2md.mcp.server`	FastMCP server for all2md document conversion.
`all2md.mcp.config`	Configuration management for MCP server.
`all2md.mcp.tools`	Tool implementations for MCP server.
`all2md.mcp.document_tools`	Document manipulation tool implementation for MCP server.
`all2md.mcp.schemas`	Tool input/output schemas for MCP server.
`all2md.mcp.security`	Security utilities for MCP server path validation.

For user-facing MCP documentation, see MCP Server.

Packagers

Submission package generators for academic publishing:

all2md.packagers.arxiv

ArXiv submission package generator.