Classify a million documents with a prompt, not a training run
LLM document classification needs no labeled training set: the labels live in the prompt. Batch turns that into a pipeline that files support tickets, routes contracts, and tags a backlog in one job.
- Zero-shot labels
- JSON output per row
- 50k docs per job
- 50% off realtime
Built for this shape of work.
Independent records, no user waiting, and a real budget: the profile the 50% batch discount is priced for.
No training set, no model hosting
Classic document classification means labeling data and training a classifier per taxonomy. An LLM classifies zero-shot from label definitions in the system prompt, and changing the taxonomy is editing text, not retraining.
Backlogs are batch-shaped
The common job is not one document, it is the archive: every ticket since 2023, every contract in the data room. That is a JSONL file, not an HTTP loop against a rate limit.
Per-token beats per-document pricing
Classification prompts are short and outputs are a label plus a confidence. On GPT-OSS 20B at batch rates a typical document costs hundredths of a cent.
One JSONL file, one job.
Each line is a complete OpenAI-compatible request with a custom_id that is echoed on the matching output row. Up to 50,000 records and 200MB per job.
{"custom_id":"ticket-8841","method":"POST","url":"/v1/chat/completions","body":{"model":"openrelay/gpt-oss-20b","messages":[{"role":"system","content":"Classify the support ticket into exactly one of: billing, bug, feature_request, account, abuse. Reply as JSON: {\"label\", \"confidence\"}."},{"role":"user","content":"I was charged twice this month and the invoice PDF 404s."}]}}
{"custom_id":"ticket-8842","method":"POST","url":"/v1/chat/completions","body":{"model":"openrelay/gpt-oss-20b","messages":[{"role":"system","content":"Classify the support ticket into exactly one of: billing, bug, feature_request, account, abuse. Reply as JSON: {\"label\", \"confidence\"}."},{"role":"user","content":"Dark mode when?"}]}}from openai import OpenAI
client = OpenAI(
base_url="https://inference.openrelay.inc/v1",
api_key="vl_••••••••", # same SDK, new base URL
)
# label every support ticket in the backlog with one job
f = client.files.create(file=open("batch.jsonl", "rb"), purpose="batch")
batch = client.batches.create(
input_file_id=f.id,
endpoint="/v1/chat/completions",
completion_window="24h", # billed at 50% of realtime
)
# poll: validating → in_progress → completed
batch = client.batches.retrieve(batch.id)
print(batch.status, batch.request_counts)
# JSONL of {custom_id, response}; failures land in error_file_id
results = client.files.content(batch.output_file_id)The right models for this job.
Batch rates are 50% off the realtime per-token catalog rates, per 1M tokens.
See the full catalogGPT-OSS 20B
OpenAI · 128K context
openrelay/gpt-oss-20b
$0.025 / $0.10
batch input / output per 1M
The price-performance pick for label-from-a-list classification. Short prompts, one-line JSON outputs.
GPT-OSS 120B
OpenAI · 128K context
openrelay/gpt-oss-120b
$0.075 / $0.30
batch input / output per 1M
Step up for nuanced taxonomies, multi-label outputs, or classification that needs a rationale per document.
Gemma 4 31B
Google · 32K context
openrelay/gemma-4-31b
$0.495 / $0.745
batch input / output per 1M
Vision input for classifying scanned pages and image-heavy PDFs by layout and content.
What separates a good run from a re-run.
Prompt and file patterns learned from real jobs, so the first submission is the one that counts.
Define labels, do not just name them
One sentence per label in the system prompt ('billing: charges, invoices, refunds...') is the difference between 80% and 95%+ agreement with human labelers. Include one edge-case example per label if you have room.
Force the closed set
Say 'exactly one of' and list the labels. Add an explicit other label instead of letting the model invent categories, then audit what lands in other to grow the taxonomy.
Ask for confidence and route on it
Have the model return a confidence value and send low-confidence rows to human review. You get a precision dial instead of a fixed error rate.
Validate on a sample first
Run 500 documents, compare against a hand-labeled sample, tune the prompt, then submit the million. Batch jobs make the full run cheap, but the iteration loop should be small.
Common questions.
Can LLMs do document classification without training data?
Yes. Zero-shot classification puts the label definitions in the prompt and asks the model to pick one. For most business taxonomies (ticket routing, contract types, content categories) a well-specified prompt on a mid-size model matches or beats a trained classifier, and changing the taxonomy takes minutes instead of a retraining cycle.
What does it cost to classify 100,000 documents?
At batch rates on GPT-OSS 20B ($0.025 per 1M input, $0.10 per 1M output tokens), a 400-token document with a 20-token label response costs about $0.0000125. 100,000 documents is roughly $1.25, submitted as two 50,000-record jobs.
How is this different from a text classification API?
Dedicated classification APIs ship fixed categories or need training data per custom label. Here the classifier is a prompt on an open model: any taxonomy, multi-label if you want it, rationale on request, and batch pricing. The output is JSON you define.
Can it classify scanned documents and PDFs?
Text-first models classify extracted text. For scans and image-heavy PDFs, run pages through DeepSeek-OCR 2 first (see the document OCR workload) or send page images to Gemma 4 31B, which accepts vision input.
How do I handle documents longer than the context window?
For classification you rarely need the whole document: the first page plus headings usually carries the signal. Truncate deterministically, or classify chunk-wise and take the majority label. Each variant is still one JSONL line.
More batch workloads
Sentiment for every review, with the why attached
Run sentiment analysis over reviews, surveys, and tickets in bulk with an LLM batch API.
Free text in, your schema out
Extract structured JSON from documents, emails, and free text with batch LLM jobs.
OCR the whole archive, priced per token instead of per page
OCR thousands of PDFs, scans, invoices, and receipts with DeepSeek-OCR 2 through a batch API.
Running agent pipelines instead of flat request files? See the agentic batch API.
Ship the first job today.
Grab an API key, upload a JSONL file, and run open models at half the realtime cost. No contract, no minimums. Deposit $5 to get $10.