Same Model, Half Price If You Can Wait 24 Hours

1. You are paying for “now”, not a smarter model

Realtime and batch endpoints often share a model name on the same price sheet. That looks like double charging. What you buy is a latency promise. Sync APIs must answer inside this HTTP request, so the vendor reserves live capacity. Batch APIs may queue work for up to 24 hours and fill spare capacity.

OpenAI’s Batch API docs state the discount in plain language: 50% cheaper than synchronous APIs, a separate and higher rate-limit pool, and a 24-hour completion window. The listed jobs are ones that can wait: evaluations, classifying large datasets, embedding content libraries, offline video work.

The usual finance miss is reading “gpt-x costs $X per million tokens” and routing nightly regression evals through the same sync client the product uses for chat. The model did not get more expensive. The multiplier did.

2. Batch is not “slow realtime”: limits and expiry differ

Batch rate limits are separate from per-model realtime limits. OpenAI says Batch usage does not consume tokens from the standard per-model pool. Hitting the live cap does not mean “this model is done for today”. Evals can change channel instead of changing model or raising the tier.

When the window closes, the batch moves to expired. Unfinished requests are cancelled. Completed responses land in the output file, and completed work is still billed. Treating Batch as all-or-nothing produces a half-finished invoice and a false “overcharge” ticket. That is the window rule, not a bug.

Submission is different too: upload JSONL, one request per line with a unique custom_id, then create the batch. A successful create only means queued. Download the output after the job ends. A tight loop against the sync SDK will not pick up the batch price by accident.

3. Anthropic: 50% on input and output, and cache can stack

Claude’s Message Batches is shorter: async Messages requests at 50% of standard API prices, higher throughput, most jobs finish in under an hour, still capped at 24 hours. The official pricing page repeats that both input and output tokens take the batch rate.

One difference matters: Anthropic says Prompt Caching discounts can stack with Message Batches. A stable system prompt written to cache, then a batch of overnight jobs, can land below “batch only, full-price prefix every time”. If every batch line mutates the prefix, the 50% applies to a full uncached prompt, and cache writes can cost more.

Do not treat the two 50% figures as interchangeable implementations. One side uses JSONL files; the other posts a request array. One stresses a separate rate-limit pool; the other stresses stacking with cache. Shared rule: can this wait.

4. What must be realtime, and what should not

Anything a user is staring at must stay realtime: support replies, editor completion, the next turn in a chat product. Moving those to Batch trades half price for an unusable product. OpenAI limits Batch to jobs that do not need an immediate response.

The other pile is where money leaks: daily eval sets, document embeddings, offline moderation, bulk rewrites, regression tests. Their deadline is “before tomorrow’s standup”, not “in 200 milliseconds”. Sync capacity is what you pay for, not a better answer.

Mixing both in one script is the expensive case. Daytime chat on realtime, then the same client, same model name, same “official unit price” spreadsheet for a hundred thousand evals at night: rate limits break first, the bill second. Two clients and two budget columns beat switching to a cheaper model.

5. Split the invoice; do not bind the card to web seats

Reconcile at least two columns: sync tokens and batch tokens. Which column grew tells you the action: interactive traffic, or wait-tolerant jobs dumped into the live pool. Folding them into “OpenAI was expensive this month” sends people to downgrade a model that should have used Batch.

API prepaid credits and ChatGPT Plus remain separate ledgers, and they should not share a card. Isolate web-seat renewals on a virtual card with a cap, such as RDVCC, so subscription renewals cannot drain eval credits. That is not a claim that a card makes Batch cheaper. The discount is the endpoint. The card only keeps seat renewals from colliding with metered API spend.