Batches, workflows and partial completion

A workflow is the thing you buy

A workflow is the customer-facing unit of work. It declares its natural unit, the output schema it returns, and the verification it runs.

A registered capability is an internal building block. It is not a purchasable service, and you never address one directly. You choose workflows.

A batch is a set of independent items

A batch is many items submitted in one request. Each item is decided on its own terms.

Item identity comes from the item_id_field you name in your input object. That value is how a returned row maps back to your source row.

Independence is the whole point. One item's refusal changes no other item's outcome, and the run continues.

Stages are the dependent case

Some workflows run as several stages, where one stage consumes what the stage before it produced. Stages therefore wait on each other.

A stage moves through BLOCKED, READY, MATERIALIZING, RUNNING, VERIFYING and AGGREGATING. It then reaches SUCCEEDED, PARTIAL, FAILED, SKIPPED or CANCELLED.

So the two shapes fail differently. An item in a batch fails alone. A stage that fails can leave every stage after it skipped.

What partial completion means

A job that accepts some items and not others reaches PARTIAL. It is not a failure, and it is not a success you can treat as complete.

Your result bundle separates the four outcomes. Accepted rows land under data/, typed failures under failures/, abstained items under abstentions/, and preserved verifier disagreements under disagreements/.

result-manifest.json is written last and is the only marker of a complete bundle. Part files on their own never mean the job finished.

Every non-success item carries a safe reason and a concrete next action. Read those before you re-run anything.

Your batch is never an execution partition

A batch describes your work. It never describes how Lithi arranges that work internally, and you gain nothing by sizing it to match anything you imagine about execution.

Size a batch by what you want back and by the bounds the workflow states. The maximum item count for a workflow is stated on its descriptor and repeated on your quote.

If a batch is too large for one submission, split it by your own data, not by a number you guessed. Each part quotes separately and reads back separately.

When a run ends PARTIAL, start with how to read a partial job.