LLM batch APIs cut prices in half, but only for traffic that can wait

How the batch discount works

Among the discounts available in LLM pricing, the batch API is the simplest: requests are submitted as a file rather than a stream, results come back within 24 hours instead of a couple of seconds, and the price is cut in half. It demands no prompt engineering, no model migration and no quality trade-off, and nearly every major provider offers it at the same number. According to the author, the real question is not whether it works but whether a 50% reduction should be the first cost lever a team reaches for — and for most teams it should not. The arithmetic below is computed from the site's catalog as verified on 11.11.2026.

50% is a ceiling, not a rate

Of the 18 models in the catalog, 16 expose a batch tier, and every one of them prices it at exactly half the real-time rate on both input and output. That uniformity makes the headline figure easy to misapply, because the discount only touches traffic that can genuinely be deferred:

effective saving = 50% × (share of calls that can wait)

• 100% batch-eligible gives 50% off
• 60% batch-eligible gives 30% off
• 20% batch-eligible gives 10% off

Few products are fully deferrable. A chatbot sits at 0% because a user is waiting; a RAG pipeline is 0% at query time, although the embedding and summarization passes behind it may not be. The workloads that genuinely qualify are the routine ones: nightly document processing, bulk classification, backfills, evals and synthetic data generation. That is why the site's Document processor archetype is the only preset that switches batch on by default.

A worked example

For 5,000 documents per day at 8,000 input and 800 output tokens each, with the engine's defaults otherwise — an 8% retry rate, a 40% cache hit rate, 15% infrastructure overhead and per-model tokenizer calibration — the monthly costs are:

The saving lands just below 50% rather than exactly on it. Retries account for the difference: a failed call still costs tokens, and the engine prices that overhead at real-time rates — a small, deliberately conservative gap.

Batch and caching do not add up

This is the part that trips people up, and it caused a bug in the authors' own engine. If caching saves roughly 90% on input and batch saves 50% on everything, it is tempting to add the two — or, more reasonably, to expect batch to take a further 50% off the already-discounted bill. Neither is what happens.

Providers stack the two discounts multiplicatively, and the batch rate applies to what is actually billed. A cached read inside a batch job costs cached_rate × 0.5, so the batch discount is taken on post-cache spend rather than on the list price:

in_spend      = in_cost - cache_savings        # what you're actually billed
batch_savings = (in_spend × 0.5 + out_cost × 0.5) × batch_pct

The consequence is unintuitive: at 100% batch eligibility the bill is exactly half, and the cache hit rate does not change that. Caching and batch are not additive levers stacking toward a deeper discount — caching decides how much is spent, and batch halves whatever that turns out to be.

A second-order effect is still worth noting. Because caching shrinks the input side of the bill, it shifts where the batch discount does its work. On this workload, Claude Sonnet 5.5's input represents 67% of the raw per-call cost, but once a 40% cache hit rate has fired, the input still being paid for is only 55% of the bill. The better the caching, the more of the batch discount effectively comes off output tokens: output-heavy work benefits more from batch than the rate card suggests, while input-heavy work with good caching benefits less.

The post acknowledges that this was initially implemented incorrectly: the engine applied the batch discount to the list input price while separately subtracting cache savings from the same tokens, discounting them twice and understating document-processor costs by 10–27%. The error was largest on the models with the steepest cache discounts. It was fixed on 31.08.2026, and the formula above is what now runs. Readers can set their own batch-eligible share and cache hit rate in the site's calculator.

Dropping a tier still beats batching

Batch is capped at 50%, but the gap between capability tiers is not. On the same workload, Claude Sonnet 5.5 with every call batched costs $2,648 per month, while Claude Haiku 5.5 at full real-time pricing — answering instantly — costs $262 per month. The cheaper tier wins by roughly 90% and returns the latency. The ordering the post recommends internalizing: pick the tier the task actually needs, get caching working, and only then decide what can wait 24 hours. Reaching for batch first means accepting a hard latency constraint to capture the smallest of the three levers.

Where batch is not on the menu

Two of the 18 models have no batch tier at all: Muse Spark Open 816B and Muse Glimmer 30B. Both are open-weights models served on inference platforms, and the post reads the omission as a product decision rather than an oversight — those platforms sell latency, so a slow, cheap queue would mean competing on the axis they deliberately declined. If batch economics are load-bearing for an architecture, those rows are eliminated regardless of where they land on price.

DeepSeek's discount is a clock, not an endpoint

One genuine exception stands out. DeepSeek has no batch API; what it offers is off-peak pricing, with every token 50% off outside the 01:00–04:00 and 06:00–10:00 UTC windows. The site models it in the batch fields because the arithmetic is identical, but the mechanism is better in one specific way: the discounted rate arrives at real-time latency, just only during certain hours, which no batch tier provides. If bulk work already runs overnight in a European or US timezone, it may already be paying the discounted rate with nothing changed.

When 50% off is actually worth it

• When the work is genuinely asynchronous — not "users would tolerate slowness", but nobody waiting at all: nightly jobs, backfills, evals, bulk classification.
• After the tier has been chosen. A tier drop usually beats the entire batch discount and costs no latency, so it should come first.
• When the eligible share is most of the volume. At 20% eligibility the gain is 10% while adding a queue, a polling loop and a failure mode — complexity that has to be paid for somewhere.
• When the tail is tolerable. The SLA is "within 24 hours", not "about an hour"; anything with a daily deadline needs a real-time fallback path, which is a second code path to maintain.
• Not as a substitute for caching. Caching cuts what is spent and batch halves it, so the first should be working before a latency constraint is accepted for the second.