all2md.roundtrip

Round-trip fidelity scoring — how much structure survives a conversion.

Where all2md.confidence asks “how much should I trust this conversion?” without a reference, this module asks a question that has one: convert a document to another format, parse it straight back, and measure what changed. The source AST is the ground truth; the re-parsed AST is the candidate:

original --render--> via-format --parse--> round-tripped
   |                                             |
   +--------------- compared -------------------+

That makes the score a genuine regression guard: a clean Markdown document round-trips through Markdown at exactly 100, so any drift is a real defect rather than measurement noise. It is also the second half of the substrate the optimize capstone consumes – all2md.confidence scores conversions where no reference exists (gnarly PDFs), this module scores the ones where a reference can be manufactured.

The comparison is deliberately structural rather than textual. Two documents that serialize to different bytes may carry identical structure (Markdown is happy to write a bullet as * or -), while two documents with identical text may have lost every heading. Five dimensions are scored independently and combined:

  • structure (weight 0.40) – the block skeleton: heading levels, list nesting and ordering, table placement, code blocks, quotes.

  • text (0.30) – the document-wide word stream, in order.

  • inline (0.15) – inline formatting: bold, italic, code, links, images.

  • tables (0.10) – table dimensions and cell text.

  • references (0.05) – hyperlink and image targets (URLs).

Dimensions absent from the source are dropped and the remaining weights are renormalized, so a document with no tables is neither rewarded nor punished for the tables it does not have.

structure and text are scored against independent alignments, and that separation is load-bearing. An earlier design aligned blocks by shape and then compared the text of each aligned pair, which meant two paragraphs could pair up merely because both were paragraphs – a demoted heading would shift every subsequent pairing and crater the text score for a document that had not lost a single word. Scoring the word stream document-wide keeps a structural change from being punished twice.

Alongside the score, the report lists concrete StructuralDelta incidents (“2 headings became paragraphs”, “table 1: 4x3 -> 4x2”) so a low score is actionable rather than merely alarming.

all2md.roundtrip.DIMENSION_WEIGHTS: dict[str, float] = {'inline': 0.15, 'references': 0.05, 'structure': 0.4, 'tables': 0.1, 'text': 0.3}

Relative weight of each fidelity dimension in the composite score.

all2md.roundtrip.TEXT_INTACT_THRESHOLD = 0.995

Text similarity at or above which the word stream is considered intact. Whitespace normalization can nudge a faithful document a hair under 1.0, so this is not exactly 1.

all2md.roundtrip.TEXT_MANGLED_THRESHOLD = 0.75

Text similarity below which lost words are a warn rather than an info.

all2md.roundtrip.TEXT_SIMILARITY_TOKEN_CAP = 20000

Combined word count (original + round-tripped) above which the order-sensitive text comparison is abandoned for an order-insensitive multiset overlap.

SequenceMatcher is quadratic in the worst case, and its worst case is not hypothetical: 10k+10k words drawn from a small vocabulary (an OCR-garbled page, a table of repeated cells) takes seconds, and 20k+20k takes half a minute. Word order is already the structure dimension’s concern, so degrading to a bag-of-words overlap above the cap costs little and bounds the runtime of all2md roundtrip on a large corpus.

class all2md.roundtrip.RoundTripReport

Bases: object

Structural fidelity of a parse -> render(via) -> parse round trip.

Parameters:

  • score (int) – Overall fidelity, 0 (nothing survived) to 100 (structurally identical).

  • band ({"high", "medium", "low"}) – Coarse bucket derived from score, using the same thresholds as ConfidenceReport.

  • source_format (str) – Format the original was parsed from (e.g. "docx").

  • via (str) – Format the document was round-tripped through (e.g. "markdown").

  • metrics (dict) – Per-dimension scores in 0-100, keyed by DIMENSION_WEIGHTS. Dimensions the source does not exercise are omitted entirely.

  • deltas (list of StructuralDelta) – Concrete differences found, most severe and most structural first.

score: int
band: Literal['high', 'medium', 'low', 'not_assessed']
source_format: str
via: str
metrics: dict[str, int]
deltas: list[StructuralDelta]
to_dict() → dict[str, Any]

Return a JSON-safe dict.

classmethod from_dict(data: dict[str, Any]) → RoundTripReport

Reconstruct a RoundTripReport from its to_dict() form.

__init__(score: int, band: ~typing.Literal['high', 'medium', 'low', 'not_assessed'], source_format: str, via: str, metrics: dict[str, int] = <factory>, deltas: list[~all2md.roundtrip.StructuralDelta] = <factory>) → None
class all2md.roundtrip.StructuralDelta

Bases: object

A single concrete difference between the original and the round trip.

Parameters:

  • kind (str) – Machine-readable category (e.g. "block_lost", "block_changed", "inline_lost", "table_changed", "reference_lost").

  • detail (str or None, default = None) – Human-readable qualifier, e.g. "heading(h2) -> paragraph".

  • count (int, default = 1) – How many times this delta occurred. Deltas sharing (kind, detail, severity) are coalesced with their counts summed.

  • severity ({"info", "warn", "error"}, default = "warn") – How serious the difference is. Purely descriptive: the score comes from the dimension metrics, not from summing delta penalties.

kind: str
detail: str | None = None
count: int = 1
severity: Literal['info', 'warn', 'error'] = 'warn'
to_dict() → dict[str, Any]

Return a JSON-safe dict, omitting detail when unset.

classmethod from_dict(data: dict[str, Any]) → StructuralDelta

Reconstruct a StructuralDelta from its to_dict() form.

__init__(kind: str, detail: str | None = None, count: int = 1, severity: Literal['info', 'warn', 'error'] = 'warn') → None
all2md.roundtrip.build_report(original: Document, roundtripped: Document, *, source_format: str, via: str) → RoundTripReport

Assemble a scored RoundTripReport from two ASTs.

all2md.roundtrip.coalesce_deltas(deltas: list[StructuralDelta]) → list[StructuralDelta]

Merge deltas sharing (kind, detail, severity), summing counts.

Keeps first-seen order, which puts document-order structural findings first.

all2md.roundtrip.net_block_deltas(deltas: list[StructuralDelta]) → list[StructuralDelta]

Cancel block_lost against block_added for the same block description.

Sequence alignment reports a moved block as one deleted and one inserted, so a document that merely gained a paragraph can be described as having lost one too. Netting the pair leaves the honest multiset statement – “one heading became a paragraph” – and lets block ordering be judged by the structure score, which is what actually measures it.

Expects already-coalesced deltas, and drops any whose count nets to zero.

all2md.roundtrip.score_roundtrip(original: Document, roundtripped: Document) → tuple[int, dict[str, int], list[StructuralDelta]]

Compare two ASTs and return (score, per-dimension metrics, deltas).

Only the dimensions the original actually exercises are scored; their weights are renormalized to sum to 1. A document with no tables therefore neither gains nor loses points for its (absent) tables, and an empty document scores a vacuous 100.

Parameters:

  • original (Document) – The ground-truth AST, parsed from the source document.

  • roundtripped (Document) – The AST re-parsed after rendering original to the intermediate format.

Returns:

The 0-100 score, the per-dimension 0-100 metrics, and the coalesced, severity-ranked structural deltas.

Return type:

tuple of (int, dict, list)