Bedrock batch inference, without the S3 scaffolding
Bedrock's batch path gives AWS-native teams half-price inference on supported foundation models, wired through S3 and IAM. This page covers how invocation jobs work, the quotas that shape real usage, and the API-key alternative when you want batch without the cloud plumbing.
AWS's batch surface in six rows.
The load-bearing facts: discount, turnaround, formats, limits, and what it takes to get access.
- Discount
- 50% of on-demand pricing for supported models
- Mechanism
- CreateModelInvocationJob over JSONL in S3; results to S3
- Format
- {recordId, modelInput} records; modelInput is each model's native schema
- Job sizing
- Per-job record minimums and per-model/region quotas apply
- Access
- AWS account, IAM role for Bedrock, S3 buckets, model access grants
- Models
- Supported foundation models; coverage varies by model and region
As publicly documented by AWS, August 2026. Verify quotas and model support per region in AWS docs.
The parts that shape real usage.
Rate cards agree everywhere (50% off is the industry number); these are the differences that decide the bill and the build.
The mechanism is infrastructure, not an API call
A Bedrock batch run means: JSONL in an S3 bucket, an IAM service role Bedrock can assume, a CreateModelInvocationJob call naming model, input, and output locations, then results and a manifest landing in S3. It is a clean design if your pipeline already lives in AWS, and a standing tax if it does not: every new team member learns the IAM posture before their first job.
Quotas and minimums shape the workflow
Jobs have minimum record counts, per-model concurrent-job quotas, and region-dependent model availability. The minimums make Bedrock batch wrong for small runs, and the quota model means production pipelines need queueing logic in front of job submission. Self-serve batch APIs invert both: no minimum, and the queue is the provider's problem.
Record format is per-model, not universal
modelInput carries each model's native request schema, so a job targeting a different model family needs different record bodies, not only a different model id. OpenAI-shaped batch surfaces normalize this: one record shape, the model field swaps.
The same request, both shapes.
Moving from Bedrock batch to an API-shaped batch surface: the record's modelInput becomes an OpenAI-shaped body, recordId becomes custom_id, and the S3/IAM layer disappears in favor of file upload over HTTPS.
{"recordId":"row-1","modelInput":{"anthropic_version":"bedrock-2023-05-31","max_tokens":1024,"messages":[{"role":"user","content":"Classify this ticket: ..."}]}}{"custom_id":"row-1","method":"POST","url":"/v1/chat/completions","body":{"model":"openrelay/gpt-oss-120b","messages":[{"role":"user","content":"Classify this ticket: ..."}]}}Stay on AWS when
- Your data, security posture, and pipelines are already AWS-native around S3
- You need a Bedrock-exclusive model in batch
- Enterprise procurement requires spend to flow through the AWS bill
Run it on OpenRelay when
- You want batch as an API call with an API key, not an IAM exercise
- Runs are sometimes small: no per-job record minimums here
- One record format across every model beats per-model schemas
AWS batch, answered.
How does batch inference work in AWS Bedrock?
You write JSONL records of {recordId, modelInput} to S3, grant Bedrock an IAM role to read and write your buckets, and call CreateModelInvocationJob with the model id and S3 locations. Bedrock processes the job asynchronously at 50% of on-demand pricing and writes results plus a manifest back to S3.
What does Bedrock batch inference cost?
Half of the model's on-demand per-token rate, for supported models. Factor in the operational side too: S3 storage, and the engineering time of the IAM/S3 wiring, which is the real cost difference versus API-shaped batch surfaces.
What are the Bedrock batch inference limits and quotas?
The ones that bite: per-job minimum record counts, caps on concurrent jobs per model, and model availability that varies by region. Check current values in the AWS quotas console for your region. If minimums or concurrency caps fight your workload shape, that is the signal to consider an API-shaped alternative.
Is there a Bedrock batch inference example without the AWS setup?
The equivalent job on OpenRelay is three SDK calls with the standard openai package: files.create on your JSONL, batches.create with a 24h window, files.content on the output file id. No buckets, roles, or model access grants; the quickstart on the Batch Inference API page is complete and runnable.
Bedrock batch vs the OpenAI Batch API: what is the difference?
Same discount, different center of gravity. Bedrock is infrastructure-shaped (S3 in, S3 out, IAM between) with per-model record schemas; OpenAI-style surfaces including OpenRelay are API-shaped (upload, create, download) with one universal record format. Choose by where your pipeline lives, not by the discount, which is 50% everywhere.
Other batch surfaces
Run the benchmark batch.
500 rows of your real workload on open models settles the pricing question in an afternoon. Deposit $5 to get $10.