Batch Inference API/ workloads

Sentiment for every review, with the why attached

Classic sentiment APIs return a polarity score. An LLM returns the sentiment, the aspects driving it, and a quotable justification, and a batch job runs that over your entire review history at classifier prices.

  • Aspect-level sentiment
  • JSON per record
  • Python via OpenAI SDK
  • 50% off realtime

Built for this shape of work.

Independent records, no user waiting, and a real budget: the profile the 50% batch discount is priced for.

Feedback arrives in piles, not streams

Survey exports, app-store reviews, NPS verbatims, quarterly ticket dumps. The natural unit of sentiment work is the whole file, which is precisely what a batch job takes as input.

Aspect-based sentiment without a bespoke model

'Negative about shipping, positive about quality' used to need a trained aspect extractor. It is now three lines of system prompt on a small open model.

Priced for exhaustive analysis

At batch rates on GPT-OSS 20B, a 150-token review with a structured verdict costs about a thousandth of a cent. Analyze everything, not a sample.

One JSONL file, one job.

Each line is a complete OpenAI-compatible request with a custom_id that is echoed on the matching output row. Up to 50,000 records and 200MB per job.

batch.jsonl · one request per linejsonl
{"custom_id":"review-33871","method":"POST","url":"/v1/chat/completions","body":{"model":"openrelay/gpt-oss-20b","messages":[{"role":"system","content":"Analyze the review. Reply as JSON: {\"sentiment\": \"positive|negative|mixed|neutral\", \"score\": -1..1, \"aspects\": [{\"aspect\", \"sentiment\"}], \"quote\"}."},{"role":"user","content":"Setup took five minutes and the dashboard is great, but support left me hanging for three days."}]}}
{"custom_id":"nps-0122","method":"POST","url":"/v1/chat/completions","body":{"model":"openrelay/gpt-oss-20b","messages":[{"role":"system","content":"Analyze the NPS verbatim. Reply as JSON: {\"sentiment\", \"score\", \"aspects\": [], \"churn_risk\": true|false}."},{"role":"user","content":"Fine I guess. Price went up again though."}]}}
batch.py · OpenAI SDK, OpenRelay base URLpython
from openai import OpenAI

client = OpenAI(
    base_url="https://inference.openrelay.inc/v1",
    api_key="vl_••••••••",          # same SDK, new base URL
)

# analyze 80,000 app-store reviews with aspect-level sentiment
f = client.files.create(file=open("batch.jsonl", "rb"), purpose="batch")

batch = client.batches.create(
    input_file_id=f.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",        # billed at 50% of realtime
)

# poll: validating → in_progress → completed
batch = client.batches.retrieve(batch.id)
print(batch.status, batch.request_counts)

# JSONL of {custom_id, response}; failures land in error_file_id
results = client.files.content(batch.output_file_id)

The right models for this job.

Batch rates are 50% off the realtime per-token catalog rates, per 1M tokens.

See the full catalog
Best value

GPT-OSS 20B

OpenAI · 128K context

openrelay/gpt-oss-20b

$0.025 / $0.10

batch input / output per 1M

The default for sentiment: fast, cheap, and reliable JSON output for polarity plus aspects.

Most popular

GPT-OSS 120B

OpenAI · 128K context

openrelay/gpt-oss-120b

$0.075 / $0.30

batch input / output per 1M

For nuance-heavy inputs: sarcasm, mixed sentiment, long-form complaints where the small model flattens detail.

What separates a good run from a re-run.

Prompt and file patterns learned from real jobs, so the first submission is the one that counts.

01

Fix the aspect vocabulary

Free-form aspects fragment ('delivery', 'shipping', 'arrival time'). List the aspects you track in the system prompt and allow other, so results aggregate cleanly in your BI tool.

02

Score on a scale, bucket later

Ask for a -1 to 1 score alongside the label. Thresholds can move in SQL after the fact; a re-run cannot be avoided if you only collected coarse labels.

03

Capture the evidence quote

Having the model return the sentence that drove its verdict makes dashboards auditable and turns 'why is shipping trending negative' into a one-query answer.

04

Track drift by re-running a fixed panel

Keep a constant 1,000-review panel in every monthly job. If the panel's scores move, your prompt or model changed; if only fresh reviews move, the sentiment did.

Common questions.

How do I run sentiment analysis from Python?

Use the OpenAI SDK against inference.openrelay.inc/v1: write one JSONL record per text, upload with files.create(purpose='batch'), then batches.create(endpoint='/v1/chat/completions'). The quickstart below is the complete script. For a handful of texts you can call chat completions directly instead of a batch.

Why use an LLM instead of a dedicated sentiment analysis API?

Dedicated APIs return polarity and stop. An LLM returns polarity plus aspect-level sentiment, churn signals, and an evidence quote, follows your exact JSON schema, and handles sarcasm and mixed sentiment better. At batch rates on a 20B model the price difference is negligible.

What does bulk sentiment analysis cost?

On GPT-OSS 20B at batch rates ($0.025 per 1M input, $0.10 per 1M output tokens), 100,000 reviews at about 150 input and 80 output tokens each cost roughly $1.20 total.

Can it do aspect-based sentiment analysis?

Yes, in the same request: list your aspects in the system prompt and require an array of {aspect, sentiment} in the JSON reply. That per-aspect breakdown is usually the whole point for product and CX teams.

Ship the first job today.

Grab an API key, upload a JSONL file, and run open models at half the realtime cost. No contract, no minimums. Deposit $5 to get $10.