Parser, Model, and API Contracts

Status: Current Last updated: 2026-10-27 (commit c0e233d0)

This page describes the boundaries between source parsing, recovery, model validation and serialization. For compiled examples, use Library Usage; for compatibility, use Versioning. A current implementation description is not a promise that every pre-1.0 API is stable.

Both parsers follow the shared model’s no-defaults/no-sentinels principle. A recovered value must represent observed data and its explicit recovery state. Parser failure must not become empty successful content or an invented required field; those substitutions cause diagnostic cascades and silent corruption.

Parser entry points

TreeSitterParser and Re2cParser are peer implementations of the shared ChatParser contract. Tree-sitter was prototyped first and is the current CLI default; it does not own the shared model. Either backend can motivate model changes, including explicit malformed or recovery states. See shared model ownership. Reuse a Tree-sitter handle when processing multiple inputs. Shared model types do not establish identical recovery, diagnostics or complete behavioral equivalence. See parser backends for supported comparisons and fragment context.

The inherent TreeSitterParser::parse_chat_file method and the streaming ChatParser trait have different result contracts:

There is no Success / Partial / Failed status field in ParseOutcome. A built model can still carry error diagnostics. Conversely, a rejected outcome does not authorize discarding diagnostics already emitted to the sink. For example, the trait word method is parse_word; the Tree-sitter parser’s inherent helper is parse_word_fragment. Do not mix their signatures.

Recovery is not admission

Recovery makes retained content available for inspection and editor diagnostics. It does not make malformed source valid. A caller must not treat is_built(), is_parsed(), or the absence of a panic as an admission proof.

parse_validated_with_parser requires validation and returns ValidChatFile under the requested ValidationPolicy. Its admission boundary checks internal failure evidence before severity filtering, then rejects parse errors or an unbuildable product before model validation. Internal failure is not a verdict that the transcript is invalid.

The optional-validation convenience APIs return a mutable ChatFile and may skip checks according to their options. Do not substitute them for required admission merely because both APIs can return a model.

The recovery design forbids inventing required text, words, enum choices or timestamps to make construction succeed. That is a requirement to verify at each producer/converter boundary, not a blanket claim that every backend already satisfies it. Inspect actual retained model and serialized output in regressions; an emitted error code alone cannot detect fabricated values. Source diagnostics must describe the original input, not invented replacements.

Bullet-text lowering likewise distinguishes a rejected body from an absent optional dependent-tier body. A MISSING continuation is reported and rejects its body; only a written continuation becomes a model segment. Low-level bullet-text and dependent-tier adapters carry ParseOutcome inside their producer-failure Result, so callers cannot construct a tier from a rejection.

Timing-tier text and its derived segments likewise have a single admission owner. TimTier::from_text classifies source or JSON text as parsed, unsupported or empty, and readers cannot mutate that classification or its segments. Unsupported text remains available for E603; empty declarations remain for E756. JSON preserves the original string and re-enters the same constructor.

Time headers use the same ownership boundary: TimeDurationValue and TimeStartValue classify their text once, with read-only segments or a copied TimeValue. Callers cannot pair out-of-range source text with valid numeric components to bypass clock assessment. Empty optional headers have an explicit absent state, so an empty start has no clock reading rather than midnight. Their JSON remains a string and re-enters the text constructor.

Source identity and coordinates

Keep the original UTF-8 source with the diagnostics that refer to it. Do not normalize line endings, trim whitespace or rebuild source from a model before using source byte offsets. Transformed text is a different coordinate space.

parse_source_with_parser binds the original source and its parse in ParsedSourceChat. Calling admit consumes that state and checks complete rules and tier alignment before returning AdmittedSourceChat, which couples unchanged source bytes with an immutable admitted model. There is no public constructor that combines arbitrary text with an unrelated model proof.

into_product releases recovery data without that admission authority; into_valid_file relinquishes the source-byte binding. A transform that removes and regenerates tiers uses its separate replacement admission, not unchanged source preservation. See the library guide.

Fragment source admission checks input length and document extent. Model rebasing is a separate consuming admission: FragmentSource::rebase and rebase_from return Result<T, FragmentRebaseError> and reject every located span outside the admitted input. This check also runs when the document origin is zero. SpanShift::try_map_spans reconstructs a complete result or returns a failure; it never lends a partly translated model to the caller. Skipped fields and independently owned diagnostic snippets retain their own coordinates.

Tree-sitter wrappers may explicitly project their own container extents and diagnostic ranges onto the caller input before translation. Payload spans are checked, not blanket-clipped. The separate in-place shift_spans_after operation retains its text-edit deletion policy and is not used for fragment translation. When a diagnostic cannot be translated, the sink reports an internal failure with the original message and snippet and no fabricated document location or labels.

For fragments, respect the API’s source origin and contextual policy. A document-relative offset is not an index into a fragment string. Legacy synthetic wrappers and native fragments are not interchangeable evidence. See the diagnostic contract for checked presentation and client coordinate conversion.

Serialization and downstream consumers

CHAT serialization writes the model’s representation. Semantic equivalence, byte-exact source preservation and validity are separate properties; state which one a test establishes. Derived word spelling is not an original source slice.

Downstream alignment and transformation code should consume typed projections and explicit admission results. Parse-health metadata helps identify recovery; it is not a replacement for whole-document validation or an acoustic-quality judgment. The JSON contract likewise distinguishes computed display fields, model structure and schema checks.

String representation is an implementation choice, not a universal storage restriction or allocation guarantee. Consult the owning types and memory architecture instead of copying layouts into integrations.

Review and verification

For a boundary change, inspect its producers, consumers and failure paths together. Keep constructor, wire-format and policy controls even when typestate prevents an invalid transition later in the pipeline. Exercise the actual public API and observe diagnostics, model content and output where each matters.

Use the spec workflow and development checks for current verification. A signature-name scan cannot prove honest recovery, complete validation or backend conformance. Compatibility promises for 1.0 remain a separate readiness requirement.

This page last changed: 2026-10-27 (commit c0e233d0). The whole book last changed: 2026-10-27 (commit 5bfee2f9).