Moderate against your policy, not a generic toxicity score
Realtime moderation gates the submit button; batch moderation handles everything else: auditing the archive after a policy change, re-screening a catalog, and QA-ing the realtime filter itself.
- Your policy in the prompt
- Severity + rationale per item
- Vision models for images
- 50% off realtime
Built for this shape of work.
Independent records, no user waiting, and a real budget: the profile the 50% batch discount is priced for.
Policy changes create instant backlogs
Every policy update makes historical content unreviewed again. Re-screening ten million items is not a realtime problem, it is one batch file per policy version.
Your policy, verbatim
Generic toxicity APIs cannot know that your marketplace bans medical claims or that your forum allows heated-but-on-topic argument. An LLM moderates against the policy text you paste into the system prompt.
Rationale on every verdict
Each output row carries the violated clause and a one-line justification, which is what appeals processes and regulator-facing audits actually need.
One JSONL file, one job.
Each line is a complete OpenAI-compatible request with a custom_id that is echoed on the matching output row. Up to 50,000 records and 200MB per job.
{"custom_id":"post-771203","method":"POST","url":"/v1/chat/completions","body":{"model":"openrelay/gpt-oss-20b","messages":[{"role":"system","content":"You are a content moderator. Policy: [1] no harassment of private individuals, [2] no medical claims, [3] no doxxing. Reply as JSON: {\"action\": \"allow|flag|remove\", \"clause\": null|1|2|3, \"severity\": 0-3, \"rationale\"}."},{"role":"user","content":"This supplement cured my diabetes in two weeks, DM me to order."}]}}
{"custom_id":"post-771204","method":"POST","url":"/v1/chat/completions","body":{"model":"openrelay/gpt-oss-20b","messages":[{"role":"system","content":"You are a content moderator. Policy: [1] no harassment of private individuals, [2] no medical claims, [3] no doxxing. Reply as JSON: {\"action\", \"clause\", \"severity\", \"rationale\"}."},{"role":"user","content":"Refs blew that call and everyone in this thread knows it."}]}}from openai import OpenAI
client = OpenAI(
base_url="https://inference.openrelay.inc/v1",
api_key="vl_••••••••", # same SDK, new base URL
)
# re-screen 2M historical posts against policy v4
f = client.files.create(file=open("batch.jsonl", "rb"), purpose="batch")
batch = client.batches.create(
input_file_id=f.id,
endpoint="/v1/chat/completions",
completion_window="24h", # billed at 50% of realtime
)
# poll: validating → in_progress → completed
batch = client.batches.retrieve(batch.id)
print(batch.status, batch.request_counts)
# JSONL of {custom_id, response}; failures land in error_file_id
results = client.files.content(batch.output_file_id)The right models for this job.
Batch rates are 50% off the realtime per-token catalog rates, per 1M tokens.
See the full catalogGPT-OSS 20B
OpenAI · 128K context
openrelay/gpt-oss-20b
$0.025 / $0.10
batch input / output per 1M
High-volume text screening: label, severity, clause, rationale at the lowest cost in the catalog.
Gemma 4 31B
Google · 32K context
openrelay/gemma-4-31b
$0.495 / $0.745
batch input / output per 1M
Vision input for image moderation and for posts where the violation lives in a screenshot or meme.
GPT-OSS 120B
OpenAI · 128K context
openrelay/gpt-oss-120b
$0.075 / $0.30
batch input / output per 1M
The escalation tier: borderline cases and nuanced policies where the small model over-flags.
What separates a good run from a re-run.
Prompt and file patterns learned from real jobs, so the first submission is the one that counts.
Number the policy clauses
Verdicts that cite clause 2 are enforceable and appealable; verdicts that cite vibes are neither. Numbered clauses in the prompt become a violated-clause field in the output.
Calibrate against your human decisions
Run the model over a thousand items your trust-and-safety team already ruled on and measure agreement before the big job. Tune the prompt where the model diverges, then scale.
Two-tier the ambiguity
Screen everything with GPT-OSS 20B, then send only its flag verdicts through GPT-OSS 120B in a second job. Most content is clearly fine; pay the larger model only for the gray zone.
Use severity to sequence the human queue
A 0-3 severity field lets reviewers work worst-first through the batch results instead of chronologically through a backlog.
Common questions.
Can I use an LLM as a content moderation API?
Yes. Put your policy in the system prompt and require a structured verdict per item. Unlike fixed-category moderation endpoints, the model enforces your rules, explains its verdicts, and updates the moment you edit the prompt. Batch is the fit for archives and re-screens; keep your low-latency filter for the submit path.
What does it cost to re-screen a large archive?
On GPT-OSS 20B at batch rates, a 200-input-token post with a 60-token verdict costs about $0.000011. Two million posts run roughly $22, cheap enough to re-screen on every policy revision instead of only after incidents.
Does it handle images and screenshots?
Route image posts to Gemma 4 31B, which takes vision input, in the same JSONL file: each line names its own model. Text-only content stays on the cheaper text model.
How accurate is LLM moderation?
Against a written policy with numbered clauses, current open models reach human-panel-level agreement on clear-cut content and disagree mostly where humans also disagree. Measure it on your own calibration set, and keep humans on appeals and on the model's own flag tier.
More batch workloads
Sentiment for every review, with the why attached
Run sentiment analysis over reviews, surveys, and tickets in bulk with an LLM batch API.
Classify a million documents with a prompt, not a training run
Classify documents, tickets, and text at scale with an LLM batch API.
LLM-as-a-judge evals, 50,000 grades per job
Run LLM-as-a-judge evaluation over thousands of outputs with one batch job.
Running agent pipelines instead of flat request files? See the agentic batch API.
Ship the first job today.
Grab an API key, upload a JSONL file, and run open models at half the realtime cost. No contract, no minimums. Deposit $5 to get $10.