> ## Documentation Index
>
> Fetch the complete documentation index at: https://lithi.ai/llms.txt
>
> Use this file to discover all available pages before exploring further.

---
title: Submit Compute work from Ray or Spark
description: >-
  Commit immutable inputs before you submit, so retries and speculative
  execution cannot duplicate paid work, and read accepted output independently
  of your runtime's partitions.
route: /docs/compute/integrations/ray-spark
page_id: docs-compute-integrations-ray-spark
page_type: howTo
content_layer: guided documentation
surface_profile: guided_docs
audience: Platform engineers submitting Compute work from a distributed runtime
voice: D
reader_question: How do I submit Compute work from Ray or Spark without paying twice for a retry?
primary_action: Commit the input, then submit one batch
source_locale: en-US
source_status: APPROVED
source_version: docs-compute-integrations-ray-spark-en-us-2026-09-05-v1
last_updated: '2026-09-05'
robots: index
claim_registry_pinned: true
claim_registry_resync: "npm run governed:tsx -- scripts/build-compute-public-claim-registry.ts --write"
---

## What this connects

Your own distributed runtime, as the caller. Your job prepares an input, submits Compute work through the Work API, and later reads the accepted bundle back.

The conformance level for the data exchange is `import_compatible`, backed by a published conformance matrix. There is no registered Ray or Spark runtime adapter, so Compute is never a compute backend for your cluster.

## Bind an exact input

Do the freezing before submission, in a stage that runs once. This is the whole design: a task that submits must be able to run twice without changing what it asks for.

Write the input, take its digest, and let the digest travel with the task:

```json
"snapshot": {
  "version_id": "<the write identity your job recorded>",
  "content_sha256": "<the digest of those exact bytes>"
}
```

Never build the request inside a task that a retry could re-execute against fresh bytes. If the bytes can move, the digest moves, and a retry stops being a retry. See [how to pin a snapshot that cannot change](/docs/compute/inputs/immutable-snapshots).

## Credentials

`input.credential_ref` names a credential you registered as a reference. Do not broadcast a secret to your workers, and do not put one in a task closure, a driver log or an environment variable your runtime prints.

A work key carries the six work scopes only. Management scopes are never mintable on it, so a submitting job cannot change identities, destinations or finance settings.

## Formats and limits

Name a released profile in `input.format` and a stable identifier in `item_id_field`. Item identity is what makes a partial result reconcilable, and your runtime's partition numbers are not identity.

The maximum item count for a workflow is stated on its descriptor and repeated on your quote. Set `deadline` and `maximum_price` in `limits`, and remember that a quote never authorizes work.

## Commit the output

Retries and speculative execution are the reason this page exists, and the protection is idempotency rather than luck.

Every effect carries an idempotency tuple and a payload digest, in separate scopes: request, batch, manifest, logical item, artifact and destination write. Two rules follow, and together they close the gap:

1. An exact replay returns the prior result. A speculative duplicate of the same task, with the same identity and the same digest, receives the original outcome rather than starting new paid work.
2. The same identity with a different digest is equivocation, and it refuses. A retry that rebuilt the request from moved bytes is rejected instead of quietly charged as a second job.

The response tells you which you received, so log it. Do not infer a replay from timing. See [how idempotency and retries work](/docs/compute/jobs/idempotency-and-retries).

Read the output independently of your partitions. `result-manifest.json` is written last and is the only complete-bundle marker; join accepted rows back by `item_id_field`, never by file order or partition index.

## What is not supported

Lithi makes no wire, SDK or endpoint compatibility claim for any distributed runtime, and does not act as a scheduler for one.

Outside the adapter:

- Running your tasks, actors or executors inside Compute.
- Submitting one request per partition and expecting them to deduplicate. Submit one batch.
- Reading a directory your job is still writing to.
- Treating a duplicate submission with fresh bytes as a safe retry. It refuses.

If a batch is partly complete, the job reports `PARTIAL` with a next action. Read the cursor, take the accepted parts, and resubmit only the outstanding items.