Synthetic data and distillation, a million examples per night
Training data is the batch workload with no realtime path at all: nobody streams a dataset. Generate instruction pairs, teacher completions for distillation, and augmentations as one JSONL job.
- Instruction pairs at scale
- Teacher runs for distillation
- Seeded, reproducible files
- 50% off realtime
Built for this shape of work.
Independent records, no user waiting, and a real budget: the profile the 50% batch discount is priced for.
Dataset generation has no latency requirement
Every record is independent and tomorrow morning is soon enough. Paying realtime rates and juggling rate limits to build a dataset is the most common way teams overspend on LLM APIs.
Distillation is a batch job by definition
Distilling a model means running the teacher over your entire prompt set and training the student on its outputs. The teacher pass is one batch file per epoch of data, at half price.
Reproducible datasets
The input file pins every prompt, sampling parameter, and model id. Regenerating the dataset after a prompt tweak is a re-submit, and dataset versions map one-to-one to job ids.
One JSONL file, one job.
Each line is a complete OpenAI-compatible request with a custom_id that is echoed on the matching output row. Up to 50,000 records and 200MB per job.
{"custom_id":"gen-00001","method":"POST","url":"/v1/chat/completions","body":{"model":"openrelay/gpt-oss-120b","messages":[{"role":"system","content":"Generate one realistic customer support conversation about the given topic. Reply as JSON: {\"messages\": [...], \"resolution\"}."},{"role":"user","content":"Topic: subscription downgrade with prorated refund"}],"temperature":1.0}}
{"custom_id":"distill-00001","method":"POST","url":"/v1/chat/completions","body":{"model":"openrelay/glm-5.2","messages":[{"role":"user","content":"Solve step by step, then give the final answer.\n\nA train leaves at 09:40 averaging 84 km/h..."}]}}from openai import OpenAI
client = OpenAI(
base_url="https://inference.openrelay.inc/v1",
api_key="vl_••••••••", # same SDK, new base URL
)
# generate 200,000 instruction pairs for fine-tuning
f = client.files.create(file=open("batch.jsonl", "rb"), purpose="batch")
batch = client.batches.create(
input_file_id=f.id,
endpoint="/v1/chat/completions",
completion_window="24h", # billed at 50% of realtime
)
# poll: validating → in_progress → completed
batch = client.batches.retrieve(batch.id)
print(batch.status, batch.request_counts)
# JSONL of {custom_id, response}; failures land in error_file_id
results = client.files.content(batch.output_file_id)The right models for this job.
Batch rates are 50% off the realtime per-token catalog rates, per 1M tokens.
See the full catalogGLM 5.2
Zhipu · 1M context
openrelay/glm-5.2
$0.91 / $2.86
batch input / output per 1M
The teacher: strongest reasoning in the catalog for generating chain-of-thought traces, hard examples, and grading passes.
GPT-OSS 120B
OpenAI · 128K context
openrelay/gpt-oss-120b
$0.075 / $0.30
batch input / output per 1M
High-volume generation with structured output support, at a fifth of the teacher's cost. The default for instruction pairs and augmentation.
GPT-OSS 20B
OpenAI · 128K context
openrelay/gpt-oss-20b
$0.025 / $0.10
batch input / output per 1M
Cheap diversity: paraphrases, negatives, and format variations where volume matters more than depth.
What separates a good run from a re-run.
Prompt and file patterns learned from real jobs, so the first submission is the one that counts.
Seed with structured variety, not temperature alone
High temperature gives you noise, not coverage. Enumerate topics, personas, difficulty levels, and formats in the prompts themselves so the file spans the distribution you want the student to learn.
Generate, then judge, then train
Chain two batch jobs: a generation pass and an LLM-as-a-judge pass that scores every example. Training only on records above a quality threshold is the cheapest dataset-quality lever there is.
Match the teacher to the skill being distilled
Reasoning traces need the strong teacher (GLM 5.2). Style, formatting, and classification behavior distill fine from GPT-OSS 120B at much lower cost. Mixing teachers per slice is one file with different model fields per line.
Deduplicate before you pay for training
Generation at scale produces near-duplicates. Embed and cluster the outputs, or add a cheap batch pass asking a small model to flag lookalikes before the fine-tune.
Common questions.
How do I generate synthetic training data with an LLM?
Write a generation prompt that specifies the format and one axis of variation (topic, persona, difficulty), expand it over your seed list into JSONL records, and submit the file as a batch. The output file is your raw dataset, typically followed by a judge pass to filter quality. The full loop is two batch jobs and no infrastructure.
What is model distillation and how does batch help?
Distillation trains a small, cheap student model on the outputs of a large teacher. The expensive part is running the teacher over the full prompt set; as a batch job that pass costs half the realtime rate and needs no rate-limit orchestration. The result file (prompt, teacher output) is your fine-tuning set.
What does a million-example dataset cost to generate?
On GPT-OSS 120B at batch rates ($0.075 per 1M input, $0.30 per 1M output tokens), a million examples at roughly 200 input and 400 output tokens each is about $135. On the GLM 5.2 teacher the same volume is about $1,300, which is why teacher-student mixes are standard.
Is synthetic data good enough to fine-tune on?
Filtered synthetic data is behind most current fine-tunes and distilled models. The failure mode is unfiltered volume: pair every generation pass with a judge pass and a dedup step, and keep a held-out human-written eval set so you measure the student against real data, not the teacher's habits.
More batch workloads
LLM-as-a-judge evals, 50,000 grades per job
Run LLM-as-a-judge evaluation over thousands of outputs with one batch job.
Free text in, your schema out
Extract structured JSON from documents, emails, and free text with batch LLM jobs.
Summarize the entire corpus, one JSONL line per document
Summarize thousands of documents, calls, and threads with one batch LLM job.
Running agent pipelines instead of flat request files? See the agentic batch API.
Ship the first job today.
Grab an API key, upload a JSONL file, and run open models at half the realtime cost. No contract, no minimums. Deposit $5 to get $10.