> ## Documentation Index > > Fetch the complete documentation index at: https://lithi.ai/llms.txt > > Use this file to discover all available pages before exploring further. --- title: Use pandas, Polars and Arrow with Compute description: >- Read an accepted result bundle with typed schemas instead of guessed ones, stream large tables rather than loading them whole, and know where a copy is actually made. route: /docs/compute/integrations/pandas-polars-arrow page_id: docs-compute-integrations-pandas-polars-arrow page_type: howTo content_layer: guided documentation surface_profile: guided_docs audience: Python data engineers reading Compute inputs and results locally voice: D reader_question: How do I read a Compute bundle in pandas, Polars or Arrow without corrupting types? primary_action: Read the manifest schema, then load the parts source_locale: en-US source_status: APPROVED source_version: docs-compute-integrations-pandas-polars-arrow-en-us-2026-09-05-v1 last_updated: '2026-09-05' robots: index claim_registry_pinned: true claim_registry_resync: "npm run governed:tsx -- scripts/build-compute-public-claim-registry.ts --write" --- ## What this connects Your own dataframe tooling, in both directions. You write an immutable bundle that Compute reads, and you read the accepted bundle Compute writes back. The conformance level is `import_compatible`: released open file formats with a published conformance matrix. These are libraries in your process, so nothing here is a network integration and no credential is involved. ## Bind an exact input Write the table once, then stop touching the file. A dataframe you are still appending to is not an input. Take the digest of the file you wrote and pin it, so the bytes you inspected are the bytes the job reads: ```bash lithi dataset validate lithi inspect ``` Validate an awkward sample first, not a clean one. Include a null of every kind you have, your longest string, your earliest and latest timestamps, and one non-ASCII value. See [how to prepare a table](/docs/compute/inputs/tabular-data). ## Credentials Nothing on this page needs one. The libraries run on your machine, and the file never leaves it until you stage it deliberately. When you do stage it, `input.credential_ref` names a storage credential you registered as a reference. A secret never appears in a request, a log, a prompt or a result file. ## Formats and limits Read the schema from the bundle, not from an inference pass. Type inference is the single largest source of silent corruption in this workflow: a decimal read as a float, an identifier read as an integer with leading zeros dropped, a timestamp read against the local zone. Load with the declared schema instead: - Read `result-manifest.json` first and take the schema it binds. - Give your reader that schema explicitly, rather than letting it guess from the first rows. - Keep identifiers as strings end to end, including on the way back out. - Write timestamps with an explicit offset, and keep one precision per column. The maximum item count for a workflow is stated on its descriptor and repeated on your quote. ## Commit the output The bundle is written for streaming, and large tables should be read that way. Iterate over `data/part-*` one part or one batch at a time, rather than concatenating everything into a single frame. Be honest with yourself about where the copies happen. Reading from a memory-mapped file into an Arrow table can avoid a copy; converting that table into a dataframe with a different memory model generally cannot, and converting to a Python object model copies again. Peak memory is set by the widest moment, not by the file size. `result-manifest.json` is written last and is the only complete-bundle marker. Check its counts against what you actually loaded, including the failure and abstention counts. ## What is not supported Lithi publishes no client library on this page, and makes no wire, SDK or endpoint compatibility claim about any dataframe project. Outside the adapter: - Executing your Python in a Compute job. Your code runs on your machine. - Reading a live dataframe, a lazy scan over a moving directory, or an open file handle. - Round-tripping a type the released format profile does not declare. - Preserving an index that is not a real column. Make it a column before you write. If a value is rejected, the refusal names the field and the row. Fix it at the source, take a new digest, then quote again with the corrected bytes.