Batch Inference API/ workloads

OCR the whole archive, priced per token instead of per page

DeepSeek-OCR 2 reads scans, PDFs, invoices, and receipts through the same batch surface as every other model. A JSONL line per page in, structured text out, at batch rates that round to cents per thousand pages.

  • DeepSeek-OCR 2
  • Scans, invoices, receipts
  • Markdown or JSON out
  • 50% off realtime

Built for this shape of work.

Independent records, no user waiting, and a real budget: the profile the 50% batch discount is priced for.

Per-page OCR pricing does not survive contact with an archive

Commercial OCR APIs price per page, which is fine for ten pages and brutal for two million. Per-token batch pricing on a model that costs about 2 cents per million tokens changes what is economical to digitize.

OCR output that is already structured

DeepSeek-OCR 2 is an LLM, so you tell it the output you want: markdown with headings preserved, or JSON with the invoice fields extracted. There is no separate parsing step after the OCR step.

One surface for OCR and what comes after

The classification, extraction, and summarization jobs that follow OCR run through the same JSONL format and API key. Chain jobs instead of chaining vendors.

One JSONL file, one job.

Each line is a complete OpenAI-compatible request with a custom_id that is echoed on the matching output row. Up to 50,000 records and 200MB per job.

batch.jsonl · one request per linejsonl
{"custom_id":"invoice-2024-0182","method":"POST","url":"/v1/chat/completions","body":{"model":"openrelay/deepseek-ocr-2","messages":[{"role":"user","content":[{"type":"text","text":"Extract this invoice as JSON: {vendor, invoice_number, date, line_items[], total}."},{"type":"image_url","image_url":{"url":"data:image/png;base64,..."}}]}]}}
{"custom_id":"scan-p014","method":"POST","url":"/v1/chat/completions","body":{"model":"openrelay/deepseek-ocr-2","messages":[{"role":"user","content":[{"type":"text","text":"Transcribe this page to markdown. Preserve headings and tables."},{"type":"image_url","image_url":{"url":"data:image/png;base64,..."}}]}]}}
batch.py · OpenAI SDK, OpenRelay base URLpython
from openai import OpenAI

client = OpenAI(
    base_url="https://inference.openrelay.inc/v1",
    api_key="vl_••••••••",          # same SDK, new base URL
)

# OCR a 40,000-page scan backlog into markdown
f = client.files.create(file=open("batch.jsonl", "rb"), purpose="batch")

batch = client.batches.create(
    input_file_id=f.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",        # billed at 50% of realtime
)

# poll: validating → in_progress → completed
batch = client.batches.retrieve(batch.id)
print(batch.status, batch.request_counts)

# JSONL of {custom_id, response}; failures land in error_file_id
results = client.files.content(batch.output_file_id)

The right models for this job.

Batch rates are 50% off the realtime per-token catalog rates, per 1M tokens.

See the full catalog
OCR

DeepSeek-OCR 2

DeepSeek · 8K context

openrelay/deepseek-ocr-2

$0.019 / $0.019

batch input / output per 1M

Purpose-built OCR model. Send page images, get faithful text with layout awareness. The cheapest tokens in the catalog.

Vision

Gemma 4 31B

Google · 32K context

openrelay/gemma-4-31b

$0.495 / $0.745

batch input / output per 1M

General vision model for pages that need interpretation beyond transcription: charts, forms with implicit structure, mixed handwriting.

What separates a good run from a re-run.

Prompt and file patterns learned from real jobs, so the first submission is the one that counts.

01

One page per record

Split PDFs to page images and give each page its own custom_id (doc-017-p03). Page-level records parallelize better, isolate failures to single pages, and keep every request well inside the context window.

02

Name the output format in the prompt

'Transcribe to markdown, preserve tables' and 'extract as JSON with these fields' are different jobs. Being explicit is what makes the output file directly loadable.

03

Render at 150-200 DPI

Higher DPI costs image tokens without improving accuracy on typical office documents. Reserve 300 DPI for small print and degraded scans.

04

Route the hard pages

Ask for a legibility flag in the output. Pages the model marks as low-confidence go to a second pass on Gemma 4 31B or to a human, and the other 99% ship.

Common questions.

Is there a DeepSeek OCR API I can call directly?

Yes. DeepSeek-OCR 2 is served on OpenRelay as openrelay/deepseek-ocr-2 through the OpenAI-compatible chat completions endpoint, realtime or batch. Send the page as an image message part and the instruction as text. No self-hosting and no GPU rental required.

What does batch OCR cost per thousand pages?

DeepSeek-OCR 2 bills at $0.02 per 1M tokens each way in batch. A typical page runs a few thousand image tokens in and under a thousand text tokens out, so a thousand pages generally lands around a dime. Compare that with per-page OCR APIs at $1 to $15 per thousand pages.

Can it extract structured fields from invoices and receipts?

Yes, in the same request. Because the OCR model is an LLM, 'extract this invoice as JSON with vendor, date, line_items, total' returns the fields directly. There is no separate template-matching or key-value post-processing step.

How do I OCR PDFs from Python?

Render each PDF page to PNG (pypdfium2 or pdf2image), base64 it into a JSONL record targeting openrelay/deepseek-ocr-2, upload the file, and create the batch with the standard OpenAI SDK pointed at inference.openrelay.inc/v1. The quickstart below is complete.

How accurate is LLM OCR versus traditional OCR engines?

On clean printed documents both are near-perfect. LLM OCR pulls ahead on layout (tables, multi-column), context (ambiguous characters resolved by meaning), and messy inputs, and it can restructure while it reads. For compliance-grade digitization, spot-check a sample per corpus, which is itself a cheap batch eval job.

Ship the first job today.

Grab an API key, upload a JSONL file, and run open models at half the realtime cost. No contract, no minimums. Deposit $5 to get $10.