all2md.transforms.pipeline

Pipeline orchestration for AST transformation and rendering.

This module provides the high-level pipeline for transforming AST documents and rendering them to markdown. It coordinates: - Transform execution with dependency resolution - Hook execution at various pipeline stages - Element-specific hooks during tree traversal - Final rendering to markdown

The pipeline is the main entry point for using transforms and hooks together.

Examples

Simple rendering:

>>> from all2md import to_ast
>>> from all2md.transforms import render
>>> doc = to_ast("document.pdf")
>>> markdown = render(doc)

With transforms:

>>> markdown = render(doc, transforms=['remove-images', 'heading-offset'])

With hooks:

>>> def log_images(node, context):
...     print(f"Image: {node.url}")
...     return node
>>> markdown = render(doc, hooks={'image': [log_images]})

Complex pipeline:

>>> from all2md.transforms import HeadingOffsetTransform
>>> markdown = render(
...     doc,
...     transforms=[HeadingOffsetTransform(offset=1), 'remove-images'],
...     hooks={
...         'pre_render': [validate_document],
...         'image': [watermark_images],
...         'post_render': [add_footer]
...     }
... )

class all2md.transforms.pipeline.Pipeline

Bases: object

Pipeline for transforming and rendering AST documents.

This class orchestrates the complete transformation and rendering pipeline, including transform resolution, hook execution, and rendering to output format.

Parameters:

• transforms (list, optional) – List of transforms to apply. Can be transform names (str) or NodeTransformer instances. Names are resolved via TransformRegistry

• hooks (dict, optional) – Dictionary mapping hook targets to lists of hook callables

• renderer (str, type, or renderer instance, optional) – Renderer to use for output. Can be: - Format name string (e.g., “markdown”) - looked up via registry - Renderer class (e.g., MarkdownRenderer) - Renderer instance (e.g., MarkdownRenderer()) Defaults to MarkdownRenderer with default options

• options (BaseRendererOptions or MarkdownOptions, optional) – Options for rendering (used if renderer is string or class, ignored if instance)

• progress_callback (ProgressCallback, optional) – Optional callback for progress updates during rendering

• strict_hooks (bool, default = False) – Enable strict mode for hook exception handling. If True, hook exceptions are re-raised and abort the pipeline. If False (default), exceptions are logged and execution continues.

Examples

Create pipeline with default markdown renderer:

>>> pipeline = Pipeline(
...     transforms=['remove-images'],
...     hooks={'pre_render': [validate]}
... )
>>> output = pipeline.execute(document)

With custom renderer:

>>> from all2md.renderers.markdown import MarkdownRenderer
>>> pipeline = Pipeline(
...     transforms=['remove-images'],
...     renderer=MarkdownRenderer(options=MarkdownRendererOptions(flavor='commonmark'))
... )
>>> output = pipeline.execute(document)

With strict hook mode:

>>> pipeline = Pipeline(
...     transforms=['remove-images'],
...     hooks={'image': [validate_image]},
...     strict_hooks=True  # Hook failures will abort pipeline
... )
>>> output = pipeline.execute(document)

Initialize pipeline with transforms, hooks, renderer, and options.

Parameters:

• transforms (list, optional) – List of transforms to apply. Can be transform names (str) or NodeTransformer instances

• hooks (dict, optional) – Dictionary mapping hook targets to lists of hook callables

• renderer (str, type, renderer instance, False, or None) –

  • str: Format name to look up via registry (e.g., “markdown”)

  • type: Renderer class to instantiate

  • instance: Pre-configured renderer to use

  • False: Skip renderer setup (for AST-only processing)

  • None: Use default MarkdownRenderer (default)

• options (BaseRendererOptions or MarkdownRendererOptions, optional) – Options for rendering (used if renderer is string or class, ignored if renderer is instance)

• progress_callback (ProgressCallback, optional) – Optional callback for progress updates during rendering

• strict_hooks (bool, default = False) – Enable strict mode for hook exception handling. If True, hook exceptions are re-raised and abort the pipeline. If False, exceptions are logged and execution continues.

__init__(transforms: list[str | NodeTransformer] | None = None, hooks: dict[Literal['post_ast', 'pre_transform', 'post_transform', 'pre_render', 'post_render', 'document', 'heading', 'paragraph', 'code_block', 'block_quote', 'list', 'list_item', 'table', 'table_row', 'table_cell', 'thematic_break', 'html_block', 'text', 'emphasis', 'strong', 'code', 'link', 'image', 'line_break', 'strikethrough', 'underline', 'superscript', 'subscript', 'html_inline', 'footnote_reference', 'footnote_definition', 'math_inline', 'math_block', 'definition_list', 'definition_term', 'definition_description'], list[Callable[[Any, HookContext], Any]]] | None = None, renderer: str | type | Any | bool | None = None, options: BaseRendererOptions | MarkdownRendererOptions | None = None, progress_callback: Callable[[ProgressEvent], None] | None = None, strict_hooks: bool = False)

Initialize pipeline with transforms, hooks, renderer, and options.

Parameters:

• transforms (list, optional) – List of transforms to apply. Can be transform names (str) or NodeTransformer instances

• hooks (dict, optional) – Dictionary mapping hook targets to lists of hook callables

• renderer (str, type, renderer instance, False, or None) –

  • str: Format name to look up via registry (e.g., “markdown”)

  • type: Renderer class to instantiate

  • instance: Pre-configured renderer to use

  • False: Skip renderer setup (for AST-only processing)

  • None: Use default MarkdownRenderer (default)

• options (BaseRendererOptions or MarkdownRendererOptions, optional) – Options for rendering (used if renderer is string or class, ignored if renderer is instance)

• progress_callback (ProgressCallback, optional) – Optional callback for progress updates during rendering

• strict_hooks (bool, default = False) – Enable strict mode for hook exception handling. If True, hook exceptions are re-raised and abort the pipeline. If False, exceptions are logged and execution continues.

get_diagnostics() → dict[str, Any]

Get diagnostic information about the pipeline configuration.

This method returns structured information about the pipeline’s configuration, useful for debugging, visualization, and documentation.

Returns:

Dictionary containing: - transforms: List of transform names in execution order - hooks: Dictionary of hook targets with their registered hooks and priorities - renderer: Renderer class name - options: Renderer options class name (if available)

Return type:

dict[str, Any]

Examples

>>> pipeline = Pipeline(
...     transforms=['remove-images', 'heading-offset'],
...     hooks={'image': [my_hook], 'pre_render': [validator]},
...     renderer='markdown'
... )
>>> diag = pipeline.get_diagnostics()
>>> print(diag['transforms'])
['RemoveImagesTransform', 'HeadingOffsetTransform']
>>> print(diag['hooks'])
{'image': [{'priority': 0, 'function': 'my_hook'}], ...}

execute(document: Document) → str | bytes

Execute complete pipeline.

This method runs the full transformation and rendering pipeline: 1. Execute post_ast hooks 2. Apply transforms (with pre/post transform hooks) 3. Apply element hooks 4. Execute pre_render hooks 5. Render to output format 6. Execute post_render hooks

If a progress_callback is configured, progress events are emitted at each stage of the pipeline.

Parameters:

document (Document) – Document to process

Returns:

Rendered output (type depends on renderer)

Return type:

str or bytes

Examples

>>> pipeline = Pipeline(transforms=['remove-images'])
>>> output = pipeline.execute(document)

class all2md.transforms.pipeline.HookAwareVisitor

Bases: NodeTransformer

Visitor that applies element hooks during tree traversal.

This visitor extends NodeTransformer to execute registered element hooks for each node type during traversal. It maintains the node path in the context for hooks that need to know the tree structure.

Parameters:

• hook_manager (HookManager) – Manager containing registered hooks

• context (HookContext) – Context to pass to hooks

Examples

>>> hook_manager = HookManager()
>>> hook_manager.register_hook('image', my_image_hook)
>>> context = HookContext(document=doc)
>>> visitor = HookAwareVisitor(hook_manager, context)
>>> processed_doc = visitor.transform(doc)

Initialize visitor with hook manager and context.

__init__(hook_manager: HookManager, context: HookContext)

Initialize visitor with hook manager and context.

transform(node: Node) → Node | None

Transform node and apply element hooks.

The node is pushed onto node_path before processing and remains there during child traversal, ensuring descendants can see full ancestry. If a hook replaces the node, the path is updated so descendants see the new node in their ancestry.

Parameters:

node (Node) – Node to transform

Returns:

Transformed node, or None if removed by hook

Return type:

Node or None

all2md.transforms.pipeline.apply(document: Document, transforms: list[str | NodeTransformer] | None = None, hooks: dict[Literal['post_ast', 'pre_transform', 'post_transform', 'pre_render', 'post_render', 'document', 'heading', 'paragraph', 'code_block', 'block_quote', 'list', 'list_item', 'table', 'table_row', 'table_cell', 'thematic_break', 'html_block', 'text', 'emphasis', 'strong', 'code', 'link', 'image', 'line_break', 'strikethrough', 'underline', 'superscript', 'subscript', 'html_inline', 'footnote_reference', 'footnote_definition', 'math_inline', 'math_block', 'definition_list', 'definition_term', 'definition_description'], list[Callable[[Any, HookContext], Any]]] | None = None, progress_callback: Callable[[ProgressEvent], None] | None = None, strict_hooks: bool = False) → Document

Apply transforms and hooks to document without rendering.

This function provides AST-only processing by applying transforms and hooks to a document without the rendering stage. It reuses Pipeline internals to maintain consistent hook execution order.

This is useful for developers who want to: - Process AST structures programmatically - Chain multiple transformation passes - Inspect/modify documents before rendering - Build custom rendering pipelines

Parameters:

• document (Document) – AST document to process

• transforms (list, optional) – List of transforms to apply. Can be transform names (str) or NodeTransformer instances

• hooks (dict, optional) – Dictionary mapping hook targets to lists of hook callables. Hook targets can be pipeline stages (‘post_ast’, ‘pre_transform’, ‘post_transform’, ‘pre_render’) or node types (‘image’, ‘link’, etc.)

• progress_callback (ProgressCallback, optional) – Optional callback for progress updates during processing

• strict_hooks (bool, default = False) – Enable strict mode for hook exception handling. If True, hook exceptions are re-raised and abort the pipeline. If False (default), exceptions are logged and execution continues.

Returns:

Processed document with transforms and hooks applied

Return type:

Document

Raises:

• TypeError – If transform is not a string or NodeTransformer

• ValueError – If transform name is not found or a hook removes the document node

Notes

The following hooks are executed in order: 1. post_ast - After AST creation (document just came from conversion) 2. pre_transform - Before each transform 3. post_transform - After each transform 4. pre_render - Before element hooks (for document-level validation) 5. Element hooks - During tree traversal (image, link, heading, etc.)

The post_render hook is NOT executed since no rendering occurs.

Examples

Apply transforms only:

>>> from all2md import to_ast
>>> from all2md.transforms import apply
>>> doc = to_ast("document.pdf")
>>> processed = apply(doc, transforms=['remove-images'])

Apply hooks only:

>>> def log_image(node, context):
...     print(f"Found image: {node.url}")
...     return node
>>> processed = apply(doc, hooks={'image': [log_image]})

Apply both transforms and hooks:

>>> from all2md.transforms import HeadingOffsetTransform
>>> processed = apply(
...     doc,
...     transforms=[HeadingOffsetTransform(offset=1), 'remove-images'],
...     hooks={
...         'pre_render': [validate_document],
...         'link': [rewrite_links]
...     }
... )

Chain multiple processing passes:

>>> doc1 = apply(doc, transforms=['heading-offset'])
>>> doc2 = apply(doc1, transforms=['remove-images'])
>>> markdown = render(doc2)

With strict hook mode:

>>> processed = apply(
...     doc,
...     hooks={'image': [validate_image]},
...     strict_hooks=True  # Hook failures will abort
... )

With progress tracking:

>>> def progress_handler(event):
...     print(f"{event.event_type}: {event.message}")
>>> processed = apply(doc, transforms=['remove-images'], progress_callback=progress_handler)

all2md.transforms.pipeline.render(document: Document, transforms: list[str | NodeTransformer] | None = None, hooks: dict[Literal['post_ast', 'pre_transform', 'post_transform', 'pre_render', 'post_render', 'document', 'heading', 'paragraph', 'code_block', 'block_quote', 'list', 'list_item', 'table', 'table_row', 'table_cell', 'thematic_break', 'html_block', 'text', 'emphasis', 'strong', 'code', 'link', 'image', 'line_break', 'strikethrough', 'underline', 'superscript', 'subscript', 'html_inline', 'footnote_reference', 'footnote_definition', 'math_inline', 'math_block', 'definition_list', 'definition_term', 'definition_description'], list[Callable[[Any, HookContext], Any]]] | None = None, renderer: str | type | Any | None = None, options: BaseRendererOptions | MarkdownRendererOptions | None = None, progress_callback: Callable[[ProgressEvent], None] | None = None, strict_hooks: bool = False, **kwargs: Any) → str | bytes

Render document with transforms and hooks using specified renderer.

This is the high-level entry point for the transformation pipeline. It creates a Pipeline instance and executes it to produce rendered output.

Parameters:

• document (Document) – AST document to render

• transforms (list, optional) – List of transforms to apply. Can be transform names (str) or NodeTransformer instances

• hooks (dict, optional) – Dictionary mapping hook targets to lists of hook callables. Hook targets can be pipeline stages (‘pre_render’, ‘post_render’, etc.) or node types (‘image’, ‘link’, ‘heading’, etc.)

• renderer (str, type, or renderer instance, optional) – Renderer to use. Can be: - Format name string (e.g., “markdown”) - looked up via registry - Renderer class (e.g., MarkdownRenderer) - Renderer instance (e.g., MarkdownRenderer()) Defaults to MarkdownRenderer

• options (BaseRendererOptions or MarkdownOptions, optional) – Options for rendering (used if renderer is string or class)

• progress_callback (ProgressCallback, optional) – Optional callback for progress updates during rendering

• strict_hooks (bool, default = False) – Enable strict mode for hook exception handling. If True, hook exceptions are re-raised and abort the pipeline. If False (default), exceptions are logged and execution continues.

• **kwargs – Additional keyword arguments passed to MarkdownOptions if options is not provided and renderer is markdown

Returns:

Rendered output (type depends on renderer)

Return type:

str or bytes

Raises:

• TypeError – If transform is not a string or NodeTransformer

• ValueError – If transform name is not found or a hook removes a required node

Examples

Basic rendering to markdown:

>>> from all2md import to_ast
>>> from all2md.transforms import render
>>> doc = to_ast("document.pdf")
>>> markdown = render(doc)

With transforms by name:

>>> markdown = render(doc, transforms=['remove-images'])

With custom renderer:

>>> from all2md.renderers.markdown import MarkdownRenderer
>>> output = render(
...     doc,
...     renderer=MarkdownRenderer(options=MarkdownRendererOptions(flavor='commonmark'))
... )

With hooks:

>>> def log_image(node, context):
...     print(f"Found image: {node.url}")
...     return node
>>> markdown = render(doc, hooks={'image': [log_image]})

Combined transforms and hooks:

>>> markdown = render(
...     doc,
...     transforms=['heading-offset', 'remove-images'],
...     hooks={
...         'pre_render': [validate_document],
...         'link': [rewrite_links],
...         'post_render': [add_footer]
...     },
...     options=MarkdownRendererOptions(flavor='commonmark')
... )

With MarkdownOptions kwargs:

>>> markdown = render(doc, flavor='gfm', emphasis_symbol='_')

With strict hook mode:

>>> markdown = render(
...     doc,
...     hooks={'image': [validate_image]},
...     strict_hooks=True  # Hook failures will abort
... )