> ## Documentation Index > > Fetch the complete documentation index at: https://lithi.ai/llms.txt > > Use this file to discover all available pages before exploring further. --- title: Turn a document corpus into cited chunks description: >- Chunk a document corpus while keeping headings, page numbers and citations intact. You get chunks and permitted vector artifacts, never a promise about retrieval quality. route: /docs/compute/examples/corpus-to-rag page_id: docs-compute-examples-corpus-to-rag page_type: tutorial content_layer: guided documentation surface_profile: guided_docs audience: Teams preparing a document corpus for their own retrieval system voice: D reader_question: How do I chunk a corpus without losing headings, page numbers or citations? primary_action: Run a twelve-document chunking test source_locale: en-US source_status: APPROVED source_version: docs-compute-examples-corpus-to-rag-en-us-2026-09-05-v1 last_updated: '2026-09-05' robots: index claim_registry_pinned: true claim_registry_resync: "npm run governed:tsx -- scripts/build-compute-public-claim-registry.ts --write" --- ## What you get Chunks that still know where they came from. - One chunk per row in `data/part-*`, carrying the document identifier, the page range and the heading path above it. - A citation on every chunk, precise enough that a person can open the source page and find the text. - Documents that could not be read in `failures/part-*`, with a typed code, a safe reason and a next action. - Vector artifacts where the workflow declares them and your data profile permits them. A chunk without a citation is unusable in an answer. If your assistant cannot show the page, nobody can check it, and an unverifiable answer is worse than no answer. ## What you need Twelve documents and a manifest. Include the formats that break naive chunkers, not just clean prose. ```text doc_id,path,format,page_count,language RAG-01,handbook/policy.pdf,pdf,42,en RAG-02,handbook/appendix-tables.pdf,pdf,11,en RAG-03,notes/faq.md,markdown,0,en RAG-04,scans/fax-notice.pdf,pdf,3,en RAG-05,handbook/policy-es.pdf,pdf,42,es RAG-06,archive/locked.pdf,pdf,0,en ``` Declare `doc_id` as your `item_id_field`. Document two is mostly tables, document four is a poor scan, and document six is protected and should be refused rather than returning empty pages. Decide your chunk bounds before you run. Name a target size and an overlap. Then name the splitting rule: split on a semantic boundary such as a heading or paragraph, never mid-sentence. ## Run it 1. Inspect locally: `lithi inspect ./corpus-fixture`. It reports page counts, encryption flags and whether a page carries a text layer at all. 2. Read the workflow: `lithi workflow explain `, and read the chunking parameters and the vector artifacts it declares. 3. Set the chunk parameters explicitly, and ask for the heading path and page range to be carried on every chunk. Retrofitting a citation later is not possible. 4. Quote it: `lithi quote --workflow --input ./corpus-fixture/manifest.csv`. The quote gives a maximum charge and an expiry, and authorizes nothing. 5. Approve the quote separately, then submit with `lithi batch submit` and track with `lithi status `. Prepare the inputs properly first: [how documents and media are accepted](/docs/compute/inputs/documents-and-media). ## Read the result Open `result-manifest.json`, then read chunks rather than counts. Take ten chunks and read them cold, without the source. Each one should make sense on its own. A chunk that starts mid-clause or ends before the condition that qualifies it will retrieve well and answer badly. Then check the boundaries you care about. No chunk should span two unrelated headings. A table should arrive as a table, not as a column of numbers with the header stranded in a different chunk. Open the cited page for five chunks and confirm the text is there. Read the scanned document separately. Lower confidence and more abstentions on a poor scan are correct. The failure to hunt for is a misread figure inside an otherwise fluent chunk. ## Where it stops You get chunks, citations and permitted vector artifacts. Nothing is loaded into your index, your database or your assistant. Good chunks are a precondition for retrieval, not a substitute for measuring it. Nothing here shows that your assistant will find the right chunk, rank it first, or answer correctly from it. Retrieval quality is a property of your index, your queries and your model. Measure it on your own questions: [grade candidates against a held-out set](/docs/compute/examples/evaluation-lab). No accuracy is promised. Extraction quality depends on the page, and a corpus mixing languages or formats will chunk unevenly. A person reviews the refusals and decides what to fix at the source.