Read the verification evidence in a result

Before you read any evidence

Verification evidence only means something once the bundle is terminal. result-manifest.json is written last, and it is the only complete-bundle marker.

A part file on its own proves nothing. Files can exist while a run is still working, and they can exist behind a run that ends in a non-success outcome.

Four checks, four different claims

Your run carries up to four kinds of checking. They are not stronger versions of one another, and passing one says nothing about the others.

• Schema validity — the output matches the declared output schema. It is a shape check only.
• Evidence closure — every claim an item makes cites the evidence the workflow required. It checks that the reasoning is supported, not that it is right.
• Semantic checks — the content satisfies rules a schema cannot express, such as a rubric gate or a reference comparison.
• Independent review — a separate check looked at the item and its answer was recorded beside the first.

Where each one lands in the bundle

verification.json records the rubric, the graders and the number of independent checks each sampled item received. provenance.json records the input, workflow, model, schema and envelope digests behind the run.

Per item, the same picture appears in the namespaced columns. lithi_verification carries what was checked, lithi_evidence carries what was cited, and lithi_confidence carries how sure the answer is.

Where independent checks did not agree, the difference is preserved under disagreements/part-* instead of being resolved quietly. Items that declined to answer land under abstentions/part-*.

Read the evidence in this order

1. Open result-manifest.json and confirm the bundle is complete.
2. Read verification.json to see what was checked, and at what rate.
3. Read failures/, abstentions/ and disagreements/ before a single accepted row.
4. Then sample accepted rows, choosing the sample by lithi_confidence and by your own impact ranking.

What a healthy run looks like

The manifest is present and binds every file, count, schema, release and retention rule. The verification rate in verification.json matches the preset you approved.

Abstentions and disagreements are present and small. Both are normal outcomes of honest checking, not defects — a run with none of either is worth a second look, not a celebration.

What none of this proves

Your preset decides how much checking runs. Standard proves structure. Verified proves that a sample survived an independent look. High assurance raises the rate, or the number of independent checks each item receives. Custom evaluation proves exactly what your rubric asks.

None of them establishes that your result is correct. A well-formed wrong answer passes schema validation cleanly, and sample evidence describes the run's tendency rather than each individual row.

Say that limit out loud when you pass the result on. State the preset, the verification rate and the sample size beside any number you quote from the run.

If the evidence looks thinner than you expected

Compare the recorded rate against the preset named in your quote. If they differ, the run did not check what you approved, and that is a support question rather than a retry.

If the rate is right but coverage still feels thin, the fix is a stronger preset on the next run. Rerunning this one changes nothing. Choosing again starts at what each quality preset proves.

Start every review with the exceptions. The reading order and what each class tells you sits in failures, abstentions and disagreements.