Give every item a stable identity

Before you start

You need one field that exists on every item and never repeats. You also need a pinned snapshot, so the run reads exactly the items you counted.

If that field holds a real customer value, you need one more thing: your own mapping table, held on your side.

Declare the field that identifies an item

The input object carries item_id_field. It names the field in your data that identifies one unit of work.

"input": {
  "item_id_field": "order_ref",
  "snapshot": { "version_id": "<version id>", "content_sha256": "<digest>" }
}

That one declaration is what every outcome points back to. An accepted row, a typed failure, an abstention and a preserved disagreement all carry it.

Results do not arrive in your input's order

A result bundle is not a rewritten copy of your input. Its outcomes are separated by kind:

data/part-*             accepted outputs
failures/part-*         typed failures and refusals
abstentions/part-*      abstained units
disagreements/part-*    preserved verifier disagreements

Item seven of your input may land in failures/, while item eight lands in data/part-0000 as the first row there. Partial completion, retries and per-item outcomes all move rows between files. Nothing promises that row n of your input is row n of anything you receive.

Your identifier survives all of that. Result columns are appended under the lithi_ prefix: lithi_status, lithi_result, lithi_error_code, lithi_confidence, lithi_evidence, lithi_verification and lithi_usage. Your own field comes back under its own name.

Join by identity, never by position

The position join is short, reads fine, and is wrong:

for record, row in zip(records, result_rows):   # wrong: position is not identity
    record.grade = row["lithi_result"]

It fails silently. Every record gets a grade, no error is raised, and the grades belong to other records. If any item abstained or failed, the offset is permanent from that row onward.

The identity join is barely longer:

by_id = {row["order_ref"]: row for row in result_rows}   # right: key on item_id_field

for record in records:
    row = by_id.get(record.order_ref)
    if row is None:
        unresolved.append(record.order_ref)
        continue
    record.grade = row["lithi_result"]

Build the same index over failures/, abstentions/ and disagreements/, and every one of your records lands in exactly one of the four.

Duplicates, and identifiers that are missing

A repeated identifier is not a duplicate row. It is two different items claiming to be the same item, and it breaks the join in both directions.

The logical item is its own idempotency scope. The same identity with the same content digest is an exact replay and returns the earlier result. The same identity with a different digest is equivocation, and it refuses rather than picking one. So deduplicate before you submit. Where two rows are genuinely different, make the identifier different too. A natural key plus an occurrence marker usually does it.

A missing identifier is simpler and worse. A row with no value in item_id_field cannot be reported against, so it is refused at validation with a safe reason and a next action. Fix it at the source and re-snapshot, rather than filling the gap with a row number.

Keep the real record on your side

Do not use a customer email address, an account number or a case reference as the item identifier. Compute never needs the real value to do the work.

Generate an opaque identifier instead. Keep the mapping from it back to the real record on your own machine, encrypted under your own key. Only you can resolve it, and the mapping never becomes part of the input, the job or the receipt.

What reconciles, and what to do when it does not

Wait for result-manifest.json. It is written last and is the only complete-bundle marker; the presence of part files never means the run finished.

Then reconcile by count. Your submitted items should equal accepted plus failed plus abstained, with disagreements recorded against items you already counted. If the totals match and every identifier resolves, the join is sound.

If an identifier is missing from all four, do not assume it succeeded. Read the job status first, because a PARTIAL job still has more to come: how to resume from a cursor.

If the job is terminal and the item is still absent, give support the item identifier and the request_id from your response envelope. Never send the input row itself.