Summarize the entire corpus, one JSONL line per document
Meeting notes, support threads, filings, call transcripts: summarization backlogs grow faster than anyone reads. A batch job turns the pile into consistent, structured summaries by tomorrow morning.
- Consistent format across docs
- 32K-1M context models
- Structured JSON summaries
- 50% off realtime
Built for this shape of work.
Independent records, no user waiting, and a real budget: the profile the 50% batch discount is priced for.
Nobody is waiting on summary #38,412
Interactive summarization is a chat feature. Corpus summarization is a pipeline: high volume, zero latency requirement, and a real budget. The 50% batch discount exists for exactly this split.
Uniform output across every document
One system prompt defines the summary shape (length, sections, JSON fields) for the whole file. Uniformity is what makes summaries loadable into search indexes, CRMs, and review tools.
Long documents fit natively
Gemma 4 31B NVFP4 covers 32K tokens per request and GLM 5.2 covers 1M, so the map step handles whole documents and the reduce step handles whole collections without exotic chunking.
One JSONL file, one job.
Each line is a complete OpenAI-compatible request with a custom_id that is echoed on the matching output row. Up to 50,000 records and 200MB per job.
{"custom_id":"call-2026-07-30-a","method":"POST","url":"/v1/chat/completions","body":{"model":"openrelay/gemma-4-31b-nvfp4-32k","messages":[{"role":"system","content":"Summarize the transcript as JSON: {\"tldr\" (2 sentences), \"decisions\": [], \"action_items\": [{\"owner\", \"task\"}], \"risks\": []}."},{"role":"user","content":"[00:00] Dana: Let's start with the Q3 capacity plan..."}]}}
{"custom_id":"thread-99120","method":"POST","url":"/v1/chat/completions","body":{"model":"openrelay/gemma-4-31b-nvfp4-32k","messages":[{"role":"system","content":"Summarize this support thread in 3 sentences: problem, resolution, follow-up owed."},{"role":"user","content":"Customer: Our webhook deliveries stopped after..."}]}}from openai import OpenAI
client = OpenAI(
base_url="https://inference.openrelay.inc/v1",
api_key="vl_••••••••", # same SDK, new base URL
)
# summarize 20,000 call transcripts into structured briefs
f = client.files.create(file=open("batch.jsonl", "rb"), purpose="batch")
batch = client.batches.create(
input_file_id=f.id,
endpoint="/v1/chat/completions",
completion_window="24h", # billed at 50% of realtime
)
# poll: validating → in_progress → completed
batch = client.batches.retrieve(batch.id)
print(batch.status, batch.request_counts)
# JSONL of {custom_id, response}; failures land in error_file_id
results = client.files.content(batch.output_file_id)The right models for this job.
Batch rates are 50% off the realtime per-token catalog rates, per 1M tokens.
See the full catalogGemma 4 31B NVFP4 32K
Google · 32K context
openrelay/gemma-4-31b-nvfp4-32k
$0.06 / $0.175
batch input / output per 1M
The volume pick: 32K context covers most documents whole, at bottom-of-catalog batch rates.
GPT-OSS 120B
OpenAI · 128K context
openrelay/gpt-oss-120b
$0.075 / $0.30
batch input / output per 1M
Higher-fidelity summaries with reliable JSON structure for executive briefs and legal-adjacent material.
GLM 5.2
Zhipu · 1M context
openrelay/glm-5.2
$0.91 / $2.86
batch input / output per 1M
1M context for the reduce step: summarize a hundred per-document summaries into one collection brief in a single request.
What separates a good run from a re-run.
Prompt and file patterns learned from real jobs, so the first submission is the one that counts.
Specify length in sentences, structure in JSON
'Summarize' alone produces wildly variable output. 'Three sentences: problem, resolution, follow-up' or an explicit JSON schema keeps 20,000 summaries interchangeable.
Map-reduce with two batch jobs
Job one summarizes each document. Job two feeds those summaries to a long-context model for the collection-level brief. Both are plain JSONL files; the second is generated from the first's output file.
Keep the source id in custom_id
custom_id is echoed on every output row. Encoding the document key there (crm-4471, call-2026-07-30-a) means results join back to your system of record with no lookup table.
Extract while you summarize
The same request can return the summary plus entities, sentiment, or action items as separate JSON fields. One pass over the corpus instead of three.
Common questions.
What is the best way to summarize thousands of documents with AI?
Batch, not a loop over a realtime API. Emit one JSONL record per document with a system prompt that pins the summary format, submit the file, and download structured summaries within the 24h window. You skip rate limits and retries and pay half the per-token rate.
How much does bulk document summarization cost?
On Gemma 4 31B NVFP4 at batch rates ($0.06 per 1M input, $0.175 per 1M output tokens), a 3,000-token document with a 200-token summary costs about $0.0002. A 100,000-document corpus is roughly $22.
How do I summarize documents longer than the context window?
Most documents fit in 32K tokens whole. For the ones that do not, chunk by section, summarize chunks in the same job, then run a short second job that merges each document's chunk summaries. GLM 5.2's 1M window usually makes the merge step a single request per document.
Can the summaries follow my exact format?
Yes. The system prompt is the format contract: field names, section order, length limits, tone. Models with structured output support (GPT-OSS family) will return valid JSON per row, ready to load into your index or CRM.
More batch workloads
OCR the whole archive, priced per token instead of per page
OCR thousands of PDFs, scans, invoices, and receipts with DeepSeek-OCR 2 through a batch API.
Sentiment for every review, with the why attached
Run sentiment analysis over reviews, surveys, and tickets in bulk with an LLM batch API.
Free text in, your schema out
Extract structured JSON from documents, emails, and free text with batch LLM jobs.
Running agent pipelines instead of flat request files? See the agentic batch API.
Ship the first job today.
Grab an API key, upload a JSONL file, and run open models at half the realtime cost. No contract, no minimums. Deposit $5 to get $10.