Read partial results with cursors

Before you start paging

You need a job that has been admitted and is producing output. A job in RUNNING may already have readable parts; a job in PARTIAL has finished with some work incomplete.

You also need somewhere to hold a cursor between calls. Cursors are how you resume without rereading, and losing one costs you a full re-page.

Take the cursor from the response, not from your own count

Each result page returns a cursor. Treat it as opaque: it is a token to hand back, not a number to parse, compare, increment or store as an offset.

Cursors are stable. Handing the same cursor back returns the same position, so a retry after a dropped connection resumes rather than duplicates.

Never build your own cursor from a row count or a timestamp. A cursor you constructed is not a cursor Compute issued, and it will not resume where you think it does.

Read the per-item outcome on every row

Each row carries its own outcome in lithi_status, with lithi_error_code set when the item did not succeed. Read them per item, not per page.

A page can contain accepted rows, failed rows and abstentions together. Treating a returned page as uniformly successful is the most common way partial reads go wrong.

Available parts are immutable, and not terminal

A part that is available to you will not change afterwards. What you read stays read, and re-reading returns the same bytes.

Immutable is not the same as complete. Available parts tell you what has landed so far; they say nothing about what is still to come, or whether the run will succeed.

result-manifest.json is written last and is the only complete-bundle marker. Until it is present, you are looking at work in progress, whatever the part files contain.

Never infer missing work from ordering

Parts are not ordered against each other, and item order inside a part is not a processing order. A gap in your source ordering is not a skipped item.

Reconcile by item identifier instead. Join each result row back to your source row through the identifier your input declared, then compare sets rather than sequences.

When the run is terminal, reconcile against the manifest counts, which cover accepted rows, failures, abstentions and disagreements together.

What success looks like

You hold a stored cursor, a set of item identifiers you have already read, and a per-item outcome for each. When the manifest appears, your counts and its counts agree.

If they do not agree, re-page from the last cursor you stored rather than from the beginning. A duplicate read is harmless; a guessed offset is not.

If paging stops partway

A dropped connection is a resume, not a rerun. Hand back your stored cursor and continue — resubmitting the job risks a second charge for work you already have.

If the cursor is rejected, the response carries a typed state and a next action. Follow that next action; do not fall back to reading from the start of a running job.

If the job itself ends PARTIAL, read the exceptions before the accepted rows. Start with failures, abstentions and disagreements, then confirm the bundle against the result bundle layout.