all2md.confidence

Conversion confidence reporting — a structured “quality card”.

Converters already compute a raft of sanity signals while parsing: how much meaningful text a PDF page yielded, whether a detected “table” was really a decorative frame (empty-cell ratio) or a table-of-contents (dot-leader ratio), how many pages had to fall back to OCR, whether an archive entry could not be parsed, and so on. Historically these were consumed for a single accept/reject decision and then discarded (surviving only as a logger.debug line).

This module captures those signals as a structured ConfidenceReport that rides on Document.metadata["confidence"]. It gives callers a reference-free read on how much to trust a conversion, and — because it emits a single 0-100 scalar score — doubles as the fitness function the optimize capstone hill-climbs on (no ground-truth reference needed).

Two kinds of evidence feed a report:

  • signals — continuous per-document metrics (meaningful_chars, chars_per_page, ocr_page_fraction, table/image counts). The PDF parser is by far the richest producer; most other formats leave these empty.

  • degraded_events — discrete incidents where the converter knowingly lost or approximated content (a table rejected as non-tabular, an archive member that would not parse, a readability/alt-text fallback taken, an OCR failure). Every parser can record these through BaseParser._record_degraded.

The report is intentionally a plain, JSON-safe structure so it serializes with the rest of the AST metadata and round-trips through ast_to_json.

all2md.confidence.dataclass(cls=None, /, *, init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False, match_args=True, kw_only=False, slots=False, weakref_slot=False)

Add dunder methods based on the fields defined in the class.

Examines PEP 526 __annotations__ to determine fields.

If init is true, an __init__() method is added to the class. If repr is true, a __repr__() method is added. If order is true, rich comparison dunder methods are added. If unsafe_hash is true, a __hash__() method is added. If frozen is true, fields may not be assigned to after instance creation. If match_args is true, the __match_args__ tuple is added. If kw_only is true, then by default all fields are keyword-only. If slots is true, a new class with a __slots__ attribute is returned.

all2md.confidence.field(*, default=<dataclasses._MISSING_TYPE object>, default_factory=<dataclasses._MISSING_TYPE object>, init=True, repr=True, hash=None, compare=True, metadata=None, kw_only=<dataclasses._MISSING_TYPE object>)

Return an object to identify dataclass fields.

default is the default value of the field. default_factory is a 0-argument function called to initialize a field’s value. If init is true, the field will be a parameter to the class’s __init__() function. If repr is true, the field will be included in the object’s repr(). If hash is true, the field will be included in the object’s hash(). If compare is true, the field will be used in comparison functions. metadata, if specified, must be a mapping which is stored but not otherwise examined by dataclass. If kw_only is true, the field will become a keyword-only parameter to __init__().

It is an error to specify both default and default_factory.

class all2md.confidence.Any

Bases: object

Special type indicating an unconstrained type.

  • Any is compatible with every type.

  • Any assumed to have all methods.

  • All values assumed to be instances of Any.

Note that all the above statements are true from the point of view of static type checkers. At runtime, Any should not be used with instance checks.

static __new__(cls, *args, **kwargs)
all2md.confidence.Band

it means the converter ran no quality instrumentation at all (no scored signals, no degraded events), so the vacuous 100 must not be read as “verified clean”. See score_conversion().

Type:

"not_assessed" is distinct from "high"

alias of Literal[‘high’, ‘medium’, ‘low’, ‘not_assessed’]

all2md.confidence.TEXT_DENSITY_MAX_PENALTY = 45.0

Points a fully text-empty page-image document can lose to the text-density penalty. Recovering that text (e.g. by enabling OCR) claws these points back, which is what makes the score usable as an optimizer fitness function.

all2md.confidence.TEXT_DENSITY_FLOOR_CPP = 200.0

Meaningful characters per page at or above which the text-density penalty is fully waived. Below it the penalty ramps linearly to TEXT_DENSITY_MAX_PENALTY.

all2md.confidence.OCR_RELIANCE_MAX_PENALTY = 12.0

Points lost when the entire document had to be recovered via OCR. OCR text is serviceable but lower-fidelity than a native text layer, so reliance on it is a mild — not catastrophic — confidence hit, scaled by the OCR page fraction.

all2md.confidence.SEVERITY_PENALTY: dict[str, float] = {'error': 10.0, 'info': 0.0, 'warn': 4.0}

Per-event penalty by severity. info events are surfaced but never scored (e.g. dropping a decorative sub-20px image is usually correct).

all2md.confidence.DEGRADED_EVENT_PENALTY_CAP = 45.0

Ceiling on the total penalty from degraded_events so a single pathological document (hundreds of unparsed archive members) cannot swamp every other signal. The score is clamped to [0, 100] regardless.

all2md.confidence.BAND_HIGH_THRESHOLD = 80

Score at or above which confidence is reported as "high".

all2md.confidence.BAND_MEDIUM_THRESHOLD = 50

Score at or above which confidence is reported as "medium" (below is "low").

all2md.confidence.SCORED_SIGNAL_KEYS: tuple[str, ...] = ('chars_per_page', 'ocr_page_fraction')

the text-density and OCR-reliance penalties read exactly these. A report carrying none of them and no degraded events was never quality-assessed – its 100 is the absence of a detector, not a clean bill of health – so it is banded "not_assessed". Keep this in sync with the signals _text_density_penalty() / _ocr_reliance_penalty() read.

Type:

Signal keys that actually feed the score

class all2md.confidence.DegradedEvent

Bases: object

A single incident where a converter knowingly lost or approximated content.

Parameters:

  • parser (str) – Short label of the producing parser (e.g. "pdf", "archive").

  • kind (str) – Machine-readable event category (e.g. "table_rejected", "unparsed_member", "readability_fallback", "ocr_failed").

  • count (int, default = 1) – How many times this event occurred. Repeated events of the same (parser, kind, detail, severity) are coalesced with their counts summed.

  • detail (str or None, default = None) – Optional human-readable qualifier (e.g. the rejection reason).

  • severity ({"info", "warn", "error"}, default = "warn") – How much the event should weigh on the score.

parser: str
kind: str
count: int = 1
detail: str | None = None
severity: Literal['info', 'warn', 'error'] = 'warn'
to_dict() → dict[str, Any]

Return a JSON-safe dict, omitting detail when unset.

classmethod from_dict(data: dict[str, Any]) → DegradedEvent

Reconstruct a DegradedEvent from its to_dict() form.

__init__(parser: str, kind: str, count: int = 1, detail: str | None = None, severity: Literal['info', 'warn', 'error'] = 'warn') → None
class all2md.confidence.ConfidenceReport

Bases: object

Structured “quality card” summarizing how much to trust a conversion.

Parameters:

  • score (int) – Overall confidence, 0 (untrustworthy) to 100 (no problems observed).

  • band ({"high", "medium", "low"}) – Coarse bucket derived from score for quick human/CLI display.

  • producer (str) – Label of the primary producing parser (e.g. "pdf").

  • signals (dict) – Continuous per-document metrics. Keys are producer-specific; common PDF keys include meaningful_chars, chars_per_page, page_count, ocr_page_fraction, tables_detected, tables_rejected, images_dropped and running_headings_demoted.

  • degraded_events (list of DegradedEvent) – Discrete lost/approximated-content incidents recorded during parsing.

score: int
band: Literal['high', 'medium', 'low', 'not_assessed']
producer: str
signals: dict[str, Any]
degraded_events: list[DegradedEvent]
to_dict() → dict[str, Any]

Return a JSON-safe dict suitable for Document.metadata['confidence'].

classmethod from_dict(data: dict[str, Any]) → ConfidenceReport

Reconstruct a ConfidenceReport from its to_dict() form.

__init__(score: int, band: ~typing.Literal['high', 'medium', 'low', 'not_assessed'], producer: str, signals: dict[str, ~typing.Any] = <factory>, degraded_events: list[~all2md.confidence.DegradedEvent] = <factory>) → None
all2md.confidence.coalesce_events(events: list[DegradedEvent]) → list[DegradedEvent]

Merge events sharing (parser, kind, detail, severity), summing counts.

Keeps first-seen order so the most significant early events lead the list.

all2md.confidence.band_for_score(score: int) → Literal['high', 'medium', 'low', 'not_assessed']

Bucket a 0-100 score into "high" / "medium" / "low".

Never returns "not_assessed": that band depends on what evidence exists, not on the score, so it is decided in score_conversion().

all2md.confidence.score_conversion(signals: dict[str, Any], events: list[DegradedEvent]) → tuple[int, Literal['high', 'medium', 'low', 'not_assessed']]

Compute the (score, band) for a conversion from its signals and events.

Reference-free: the score starts at 100 and subtracts a text-density penalty, an OCR-reliance penalty, and a (capped) degraded-event penalty. The result is clamped to [0, 100].

A conversion that produced no scored signals and no degraded events is banded "not_assessed" rather than "high". Formats without quality instrumentation (docx, pptx, html) hit this path: their score is a vacuous 100 that means “no detector ran”, not “verified clean”, and conflating the two would let a mangled .docx report 100/HIGH.

Parameters:

  • signals (dict) – Continuous per-document metrics (see ConfidenceReport).

  • events (list of DegradedEvent) – Discrete degraded-content incidents.

Returns:

The integer score and its confidence band.

Return type:

tuple of (int, str)

all2md.confidence.build_report(producer: str, signals: dict[str, Any], events: list[DegradedEvent]) → ConfidenceReport

Assemble a scored ConfidenceReport from raw signals and events.

Events are coalesced (see coalesce_events()) before scoring so repeated incidents count once with an aggregate count.