all2md.cli.processors

Specialized processing functions for all2md CLI.

This module contains focused processing functions extracted from the main() function to improve maintainability and testability.

all2md.cli.processors.extract_sections_from_document(doc: Document, extract_spec: str) → Document

Extract specific sections from a document based on extraction specification.

This function is a thin wrapper around the extract_sections() utility from document_utils, maintained for backward compatibility.

Parameters:

  • doc (Document) – Source document to extract sections from

  • extract_spec (str) – Extraction specification: - Name pattern: “Introduction”, “Intro*”, “Results” (uses fnmatch) - Single index: “#:1” (1-based) - Range: “#:1-3” (1-based, inclusive) - Multiple: “#:1,3,5” (1-based) - Open-ended: “#:3-” (from 3 to end)

Returns:

New document containing only extracted sections, with ThematicBreak separators

Return type:

Document

Raises:

ValueError – If extraction spec is invalid or no matching sections found

Examples

>>> doc = Document(children=[...])
>>> extracted = extract_sections_from_document(doc, "Introduction")
>>> extracted = extract_sections_from_document(doc, "#:1-3")
>>> extracted = extract_sections_from_document(doc, "Chapter*")
all2md.cli.processors.generate_outline_from_document(doc: Document, max_level: int = 6, *, line_numbers: bool = False, rendered_markdown: str | None = None) → str

Generate a markdown-formatted outline from document headings.

Parameters:

  • doc (Document) – Source document to extract outline from

  • max_level (int) – Maximum heading level to include (1-6, default: 6)

  • line_numbers (bool, default False) – If True, prefix each heading with the 1-based line number it occupies in the rendered Markdown (see rendered_markdown). This gives callers a heading -> line map to feed back into --extract line:X-Y.

  • rendered_markdown (str, optional) – Markdown rendering of doc, used to resolve line numbers. Required when line_numbers is True.

Returns:

Markdown-formatted outline with nested list structure

Return type:

str

Examples

>>> doc = Document(children=[...])
>>> outline = generate_outline_from_document(doc, max_level=3)
>>> print(outline)
* Introduction
  * Background
    * Related Work
* Methods
  * Data Collection
With line numbers:
>>> print(generate_outline_from_document(doc, line_numbers=True, rendered_markdown=md))
 1: * Introduction
 5:   * Background
12: * Methods
all2md.cli.processors.is_line_extract_spec(extract_spec: str | None) → bool

Return True when an –extract spec selects by output line range.

all2md.cli.processors.parse_slice_spec(spec: str) → Tuple[int, int]

Parse a --slice X/Y spec into 1-based (x, y).

Raises:

ValueError – If the spec is malformed or out of the 1 <= x <= y range.

all2md.cli.processors.line_selection_from_args(args: Namespace) → str | None

Derive a normalized line-selection string from --head/--tail/--lines.

Returns None when none of the flags are set. Only one is expected to be set at a time (enforced by validation); if several are present, the first of head/tail/lines wins.

class all2md.cli.processors.TransformSpec

Bases: TypedDict

Serializable specification for reconstructing a transform instance.

name: str
params: Dict[str, Any]
all2md.cli.processors.prepare_options_for_execution(options: Dict[str, Any], input_path: Path | None, parser_hint: str, renderer_hint: str | None = None) → Dict[str, Any]

Prepare CLI options for API consumption based on detected formats.

all2md.cli.processors.load_converter_config_options(*, explicit_path: str | None, no_config: bool) → Dict[str, Any]

Return converter options from the config file for view/serve.

Standalone subcommands like view and serve only convert documents; they don’t expose the converter’s per-format flags. This reuses the main converter’s config machinery so a single config file drives all2md, view, and serve identically – e.g. [pdf] detect_columns = true or a top-level attachment_mode applies everywhere.

The result is a dot-notation options dict (e.g. {"pdf.detect_columns": True, "attachment_mode": "save"}) ready for prepare_options_for_execution(). The subcommand flag sections ([view], [serve], …) and the [rich] terminal-styling table are stripped so they never leak in as converter options.

Parameters:

  • explicit_path (str or None) – Explicit config path (typically from a --config flag).

  • no_config (bool) – If True, skip config loading and return an empty dict.

Returns:

Flattened converter options, or an empty dict when no config applies.

Return type:

dict

all2md.cli.processors.build_transform_instances(parsed_args: Namespace) → list | None

Build transform instances from CLI arguments.

Parameters:

parsed_args (argparse.Namespace) – Parsed CLI arguments

Returns:

List of transform instances (in order) or None if no transforms

Return type:

Optional[list]

Raises:

argparse.ArgumentTypeError – If transform is unknown or required parameters are missing

all2md.cli.processors.apply_security_preset(parsed_args: Namespace, options: Dict[str, Any]) → Dict[str, Any]

Apply security preset configurations to options.

Parameters:

  • parsed_args (argparse.Namespace) – Parsed command line arguments

  • options (Dict[str, Any]) – Current options dictionary

Returns:

Updated options with security presets applied

Return type:

Dict[str, Any]

Notes

Security presets set multiple options to secure defaults using format-qualified keys to prevent ambiguity and unintended side effects. Explicit CLI flags can still override preset values if specified.

Format-qualified keys ensure precise targeting: - html.* -> HtmlOptions fields - html.network.* -> HtmlOptions.network (NetworkFetchOptions) - html.local_files.* -> HtmlOptions.local_files (LocalFileAccessOptions) - mhtml.local_files.* -> MhtmlOptions.local_files (LocalFileAccessOptions) - eml.html_network.* -> EmlOptions.html_network (NetworkFetchOptions) - max_attachment_size_bytes -> BaseParserOptions (top-level, no prefix)

all2md.cli.processors.setup_and_validate_options(parsed_args: Namespace) → Tuple[Dict[str, Any], Literal['auto', 'archive', 'asciidoc', 'ast', 'bbcode', 'chm', 'csv', 'docx', 'dokuwiki', 'eml', 'enex', 'epub', 'fb2', 'html', 'ini', 'ipynb', 'jinja', 'json', 'latex', 'markdown', 'mbox', 'mediawiki', 'mhtml', 'odp', 'ods', 'odt', 'openapi', 'org', 'outlook', 'pdf', 'plaintext', 'pptx', 'rst', 'rtf', 'sourcecode', 'textile', 'toml', 'webarchive', 'xlsx', 'yaml', 'zip'], list | None]

Set up conversion options and build transforms.

Parameters:

parsed_args (argparse.Namespace) – Parsed command line arguments

Returns:

Tuple of (options_dict, format_arg, transforms)

Return type:

Tuple[Dict[str, Any], DocumentFormat, Optional[list]]

Raises:

argparse.ArgumentTypeError – If config file cannot be loaded or transform building fails

all2md.cli.processors.process_multi_file(items: List[CLIInputItem], parsed_args: Namespace, options: Dict[str, Any], format_arg: str, transforms: list | None = None) → int

Process multiple files with appropriate output handling.

Parameters:

  • items (List[CLIInputItem]) – List of CLI input items to process

  • parsed_args (argparse.Namespace) – Parsed command line arguments

  • options (Dict[str, Any]) – Conversion options

  • format_arg (str) – Format specification

  • transforms (list, optional) – List of transform instances to apply

Returns:

Exit code (0 for success, highest error code otherwise; see constants.py for complete list)

Return type:

int

all2md.cli.processors.load_options_from_json(json_file_path: str) → dict

Load options from a JSON file.

Parameters:

json_file_path (str) – Path to the JSON file containing options

Returns:

Dictionary of options loaded from the JSON file

Return type:

dict

Raises:

argparse.ArgumentTypeError – If the JSON file cannot be read or parsed

all2md.cli.processors.merge_exclusion_patterns_from_json(parsed_args: Namespace, json_options: dict) → List[str] | None

Merge exclusion patterns from JSON options if not specified via CLI.

Parameters:

  • parsed_args (argparse.Namespace) – Parsed command line arguments

  • json_options (dict) – Options loaded from JSON file

Returns:

Updated exclusion patterns or None if no changes

Return type:

Optional[List[str]]

all2md.cli.processors.parse_merge_list(list_path: Path | str, separator: str = '\t') → List[Tuple[Path, str | None]]

Parse merge list file and return file paths with optional section titles.

Parameters:

  • list_path (Path or str) – Path to the list file containing documents to merge, or ‘-’ to read from stdin

  • separator (str, default 't') – Separator character for the list file (default: tab for TSV)

Returns:

List of (file_path, section_title) tuples where section_title is None if not specified in the list file

Return type:

List[Tuple[Path, Optional[str]]]

Raises:

argparse.ArgumentTypeError – If the list file cannot be read or contains invalid entries

Notes

List file format: - TSV format: path[<separator>section_title] - Lines starting with # are comments - Blank lines are ignored - File paths are resolved relative to the list file directory (or cwd if stdin) - If no section title is provided, None is used - Use ‘-’ as the path to read the list from stdin

Examples

List file content:

# This is a comment
chapter1.pdf    Introduction
chapter2.pdf    Background
chapter3.pdf

Results in:

[
    (Path('chapter1.pdf'), 'Introduction'),
    (Path('chapter2.pdf'), 'Background'),
    (Path('chapter3.pdf'), None)
]

Reading from stdin:

$ echo "chapter1.pdf\\tIntro" | all2md --merge-from-list - --out book.md
all2md.cli.processors.process_merge_from_list(args: Namespace, options: Dict[str, Any], format_arg: str, transforms: list | None = None) → int

Process files from a list file and merge them into a single document.

Parameters:

  • args (argparse.Namespace) – Command-line arguments containing merge-from-list settings

  • options (Dict[str, Any]) – Conversion options

  • format_arg (str) – Format specification

  • transforms (list, optional) – List of transform instances to apply

Returns:

Exit code (0 for success, highest error code otherwise)

Return type:

int

all2md.cli.processors.process_dry_run(items: List[CLIInputItem], args: Namespace, format_arg: str) → int

Show what would be processed without performing any conversions.

This is a backward-compatible wrapper that delegates to the implementation in _processors_detect.py.

all2md.cli.processors.process_files_collated(items: List[CLIInputItem], args: Namespace, options: Dict[str, Any], format_arg: str, transforms: list | None = None) → int

Collate multiple inputs into a single output using an AST pipeline.

all2md.cli.processors.process_files_with_splitting(items: List[CLIInputItem], args: Namespace, options: Dict[str, Any], format_arg: str, transforms: list | None = None) → int

Process files with document splitting into multiple output files.

Parameters:

  • items (List[CLIInputItem]) – Input items to process

  • args (argparse.Namespace) – CLI arguments including split_by specification

  • options (Dict[str, Any]) – Conversion options

  • format_arg (str) – Source format specification

  • transforms (list, optional) – AST transforms to apply

Returns:

Exit code (0 for success)

Return type:

int

all2md.cli.processors.generate_output_path(input_file: Path, output_dir: Path | None = None, preserve_structure: bool = False, base_input_dir: Path | None = None, dry_run: bool = False, target_format: str = 'markdown') → Path

Generate output path for a converted file.

Parameters:

  • input_file (Path) – Input file path

  • output_dir (Path, optional) – Output directory

  • preserve_structure (bool) – Whether to preserve directory structure

  • base_input_dir (Path, optional) – Base input directory for preserving structure

  • dry_run (bool, default=False) – If True, don’t create directories

  • target_format (str, default="markdown") – Target output format to determine file extension

Returns:

Output file path

Return type:

Path

all2md.cli.processors.convert_single_file(input_item: CLIInputItem, output_path: Path | None, options: Dict[str, Any], format_arg: str, transforms: list | None = None, show_progress: bool = False, target_format: str = 'markdown', transform_specs: list[TransformSpec] | None = None, progress_callback: Any | None = None, extract_specs: List[str] | None = None, outline: bool = False, outline_max_level: int = 6, line_numbers: bool = False, slice_spec: str | None = None, line_select: str | None = None) → Tuple[int, str, str | None]

Convert a single file to the specified target format.

Parameters:

  • input_item (CLIInputItem) – CLI input item describing the source

  • output_path (Path, optional) – Output file path. If None, prints to stdout (markdown only)

  • options (Dict[str, Any]) – Conversion options

  • format_arg (str) – Source format specification

  • transforms (list, optional) – List of transform instances to apply

  • show_progress (bool, default False) – Whether to show progress (currently unused)

  • target_format (str, default 'markdown') – Target output format (e.g., ‘markdown’, ‘docx’, ‘pdf’, ‘html’)

  • transform_specs (list[TransformSpec], optional) – Serializable transform specifications to rebuild in worker processes

  • progress_callback (ProgressCallback, optional) – Optional callback for progress updates

  • extract_specs (list[str], optional) – One or more extraction selectors (e.g., [“Introduction”, “table:1”]). See _extraction_output() for the supported grammar.

  • outline (bool, default False) – Whether to output document outline instead of full content

  • outline_max_level (int, default 6) – Maximum heading level to include in outline (1-6)

  • line_numbers (bool, default False) – Whether to annotate Markdown output with line numbers (outline headings, extracted lines, or every line of a full conversion)

  • slice_spec (str, optional) – --slice X/Y specification; emit the Xth of Y semantic slices.

  • line_select (str, optional) – Normalized --head/--tail/--lines selection (e.g. “head:10”).

Returns:

(exit_code, file_path_str, error_message)

Return type:

Tuple[int, str, Optional[str]]

all2md.cli.processors.process_files_unified(items: List[CLIInputItem], args: Namespace, options: Dict[str, Any], format_arg: str, transforms: list | None = None) → int

Process CLI inputs with unified progress handling.