Run a stream as bounded windows

Before you start

Compute does not open an unbounded continuous session over a stream. Every run is bounded, so the work you do here is turning a stream into a sequence of finished windows.

Each window becomes its own job, with its own quote, its own approval and its own result bundle. You need a source that can name a position: offsets and partitions, a monotonic watermark column, or a commit or transaction identifier.

Pin the window at both ends

Write down the start and the end, and say once whether each end is inclusive. An open-ended window is not a window.

Then materialize it. Read the range out to a file or a manifest, take its digest, and pin that as your snapshot. A live topic or a live table is not a frozen source, however precisely you describe the range you meant.

"snapshot": {
  "version_id": "orders-2026-09-04T00-2026-09-05T00",
  "content_sha256": "<the digest of the materialized window>"
}

Keep the naming regular. A window identifier you can regenerate from its bounds is what lets you find a run again a month later.

Decide the late-arrival policy before you run

Records arrive after the moment they describe. Choose how you handle that before the first window, not after a number looks wrong.

Two policies are honest. You can hold a window open for a stated lateness allowance, closing it only once that allowance has passed, which trades freshness for completeness. Or you can close on time and route late records into the next window, or into a dedicated correction window.

What you must not do is reopen a closed window. It has been quoted, approved, paid for and joined, and rewriting it changes results you have already acted on.

Identify occurrences, not arrivals

A stream re-delivers. The same record can appear twice with no defect anywhere, so an occurrence needs a stable identity of its own.

Build that identity from the source position and the record key — a partition and offset, or a key and a sequence number. Never build it from arrival order.

The logical item is its own idempotency scope. The same identity with the same content digest is an exact replay, and it returns the earlier result. The same identity with a different digest is equivocation, and it refuses. Deduplicate within the window before you submit, and join every result back by identity rather than by position: see how item identity works.

Run each window as its own job

Quote the window, approve it, run it, and collect its bundle before you open the next one. That keeps every cost and every result attributable to one bounded range of your stream.

If you drive windows on a schedule, the occurrence key is derived from the schedule, its version and the intended occurrence. A re-fired occurrence is therefore recognized as the same intended occurrence, so a replay does not become a second paid run.

What success looks like

Every window has exactly one job, one result bundle and one receipt reference. Wait for result-manifest.json in each bundle, because it is written last and is the only complete-bundle marker.

Counts should reconcile per window, not across the stream. Submitted items equal accepted plus failed plus abstained, for that window alone.

If you find a gap or a late batch

Run a correction window over the missed range as a new job. Give it its own bounds and its own identifier, so it is visible as a correction rather than hidden inside a widened window.

If a scheduled occurrence never produced a job, do not submit the range by hand and risk a duplicate. Start from what to do when a schedule is missed.