Compute
Read failures, abstentions and disagreements first
Review every failure, abstention, disagreement, low-confidence and high-impact item before you sample accepted rows, and know what each outcome says about your data or the workflow.
Read the exceptions before the accepted rows
Start with what did not go cleanly. The exceptions are small, they are where the information is, and they change how you read everything else.
Confirm the bundle first. result-manifest.json is written last and is the only complete-bundle marker, so an exception count taken before it exists is not a count of anything.
The reading order
failures/part-*— every typed failure and refusal.abstentions/part-*— every unit that declined to answer.disagreements/part-*— every preserved disagreement between independent checks.- Accepted rows with the lowest
lithi_confidence. - Accepted rows you have ranked as high impact, whatever their confidence.
- A declared random sample of everything else.
Work down that list. Steps one to five are read in full; only step six is sampled.
What each class tells you
| Path | Usually tells you about | Typical next move |
|---|---|---|
failures/ | Your input, your request or a declared limit | Fix the input or the request, then rerun only those items |
abstentions/ | Evidence in your data that was not sufficient | Supply better evidence, or accept the abstention as the honest answer |
disagreements/ | The boundary of the workflow or your rubric | Decide the ambiguous cases yourself, then tighten the rubric |
Low lithi_confidence | Items near the workflow's limit on your data | Review by hand before anything downstream reads them |
Failures are mostly about you. Abstentions are mostly about your data. Disagreements are mostly about the workflow's definition meeting your material.
Failures and refusals
Each failed item carries lithi_status, a typed lithi_error_code, a safe reason and a next action. Group them by code before you read them one by one.
One code across thousands of items is a systematic problem with the request or the source. Many codes scattered thinly is a data-quality problem in the input.
Never treat a failure as free. Read the applicable commercial policy for what a failed attempt is charged, and rerun only the affected items rather than the whole job.
Abstentions
An abstention is a refusal to guess. It means the workflow found the evidence insufficient for the answer it was asked to produce.
That is a useful result, not a defect. A confident wrong answer in the same slot would cost you more, and would be far harder to find.
Read them for a pattern. If a whole category abstains, the input for that category is missing something the workflow needs, and no rerun fixes it.
Disagreements
Independent checks that reached different answers are preserved rather than resolved quietly. You see the disagreement itself, not a tidied summary.
Concentrated disagreements mark a genuinely ambiguous class in your material. That is a signal to sharpen your own definition, in a rubric, before the next run.
If disagreements are absent entirely, check the preset recorded in verification.json. Standard runs no independent review, so silence there is expected rather than reassuring.
Declare your sampling policy before you sample
Write down the sample rate, how rows are selected, who reviews them and what result would make you reject the run. Do it before you look at any accepted row.
An undeclared sample becomes a search for reassurance. A declared one can fail, which is the only reason it is worth running.
Record the policy beside the result, with the preset and verification rate from verification.json. Anyone quoting your numbers later needs all three.
What success looks like
Every failure, abstention and disagreement has been read and dispositioned. Your low-confidence and high-impact rows have been reviewed by a person.
Your sample came back inside the threshold you declared in advance, and you can state the limits of what the run proves. Then, and only then, the accepted rows are safe to use.
Where the checking itself needs explaining, start with what the verification evidence proves. To rerun only the affected items, see submitting work.