An AI inference platform for open models
OpenRelay serves open-weight models behind an OpenAI-compatible API billed per token, or on a dedicated endpoint of your own at one flat monthly rate. When you need your own runtime, the same account rents GPUs by the hour.
- OpenAI-compatible API
- From $0.035 per 1M input tokens
- Dedicated endpoints at a flat monthly rate
- Standard batch at 50% off
- GPUs from $0.18/hr
Four ways to run a model.
They differ in who runs the model and what you pay for. One account and one API key cover all four.
Sync API
Call a catalog model in real time, pay per token, and pay nothing while idle.
Serverless inferenceBatch API
Upload a JSONL file of requests and collect the results within 24 hours; standard batches bill at half the Sync rate. Async, which queues single requests, bills 20% off Sync within 1 hour and 50% off Sync within 24 hours.
Batch inferenceDedicated endpoint
A model on capacity reserved for your organization, tuned for your throughput and latency, at one flat monthly rate.
Get a quoteYour own server
Run your weights and serving stack on a GPU VM or a Pod, billed hourly while it runs.
GPU pricingThe models and their rates.
Read from the catalog when this page loads. A standard batch bills each request at half.
Inference pricing| Model | Input / 1M | Output / 1M | ||
|---|---|---|---|---|
| Gemma 4 31Bopenrelay/gemma-4-31b | 256K | $0.07 | $0.30 | Vision-grounded chat |
| GLM 5.3openrelay/glm-5.3 | 256K | $0.079 | $3.00 | Reasoning, coding, agents |
| GLM 5.3 Flashopenrelay/glm-5.3-flash | 256K | $0.045 | $0.18 | Agentic coding, long context, vision |
| DeepSeek V4.1-Flashopenrelay/deepseek-v4.1-flash | 384K | $0.035 | $0.33 | Agents, coding, vision, long context |
| Kimi K3.1 Proopenrelay/kimi-k3.1-pro | 256K | $0.24 | $7.00 | Agents, coding, vision, long context |
| DeepSeek-OCR 2openrelay/deepseek-ocr-2 | 8K | $0.039 | $0.039 | Document to markdown, OCR, data extraction |
| BGE-M3openrelay/bge-m3 | 8K | $0.013 | Input only | Embeddings |
What the gateway does per request.
Every call to inference.openrelay.inc passes through one gateway before it reaches a model.
How routing worksAuthenticates the key
An or_ key, sent as a Bearer token or, for Anthropic clients, as x-api-key.
Speaks your SDK's format
OpenAI Chat Completions and Embeddings, and Anthropic Messages on models that support it.
Retries before the first byte
Where a model has more than one backend, a failure before the response starts moves the request to the next.
Prices cached prompts separately
Prompt tokens served from cache bill at the model's cached-input rate.
Bills what it reports
Every response carries the token counts you pay for. A failed request is not billed.
Checks the balance first
Requests draw on a prepaid balance and are refused once it falls below the floor.
Which one fits.
Pick by who is waiting for the answer and whose model it is.
| Workload | Use | |
|---|---|---|
| Chat, agents and RAG on a catalog model | Sync API | Input and output tokens |
| Evals, labeling, backfills nobody waits on | Batch API | Tokens at half the Sync rate on the standard tier |
| Steady traffic that needs capacity of its own | Dedicated endpoint | One flat monthly rate for the throughput you reserve |
| Your own weights, fine-tune or runtime | GPU VM or Pod | An hourly rate while it runs |
We do not sell scale-to-zero hosting for your own model. The serverless GPU page covers when it pays and what to use here instead.
Two lines change.
Existing OpenAI code runs against OpenRelay with a new base URL and an OpenRelay key.
Quickstartfrom openai import OpenAI
client = OpenAI(
base_url="https://inference.openrelay.inc/v1",
api_key="or_••••••••",
)
reply = client.chat.completions.create(
model="openrelay/gemma-4-31b",
messages=[{"role": "user", "content": "Summarize this ticket in one line."}],
)
print(reply.choices[0].message.content)AI inference platforms, answered.
What is an AI inference platform?
The layer between a trained model and the application calling it: GPUs with the model loaded, a server that schedules requests onto them, an API in front and metering behind. Using one means you send requests while someone else keeps the model deployed.
Which models does OpenRelay serve?
Models on the API include Gemma 4 31B, GLM 5.3, GLM 5.3 Flash, DeepSeek V4.1-Flash, Kimi K3.1 Pro, DeepSeek-OCR 2, and BGE-M3. The table above has each one's context window and rates, and the inference catalog has the details.
How is OpenRelay priced?
API calls bill per token at each model's input and output rates, from $0.035 per 1M input tokens, plus a cached-input rate on models that have one. That is the Sync rate. Async bills 20% off Sync within 1 hour and 50% off Sync within 24 hours; standard batches bill half the Sync rate and expedited batches the full rate. GPU VMs and Pods bill an hourly rate, from $0.18/hr, for the time they run. All of it draws on one prepaid balance.
What is a dedicated inference endpoint?
A model served on capacity reserved for your organization, called through the same API with its own private model id. We tune it to your workload for throughput, latency and time to first token, and bill one flat monthly rate: up to the throughput you reserve, every request is already paid for. Only your organization can call it.
Can I serve my own model?
Not on the shared API, which serves the catalog. Run your weights on a GPU VM or a Pod with the serving stack you choose, or ask us about a dedicated endpoint.
Do I have to change my code?
Not for OpenAI clients: set the base URL to https://inference.openrelay.inc/v1 and use an OpenRelay key. Anthropic clients work the same way on models that support the Messages API.
What does an idle hour cost?
Nothing on the API, which has no hourly charge. A GPU VM or Pod bills while it runs, so stop it when it sits idle: stopped time bills only disk storage.
One key for every model.
Make a deposit, call any model in the catalog, and rent a GPU from the same account when you need your own.