That has more to do with geopolitics than it does the current price of memory.

No. Prices will crash when supply side expands to meet the increased demand. Because demand won't go down to pre-bubble times any time soon. Unfortunately the supply side has been very slow in increasing production, partly because most steps of the production chain are all maxed out.

On a long enough scale you are right that prices will likely normalize to a better level, but before 2030? That would mean the factories are built quickly once they begin.

The entire reason why hardware prices are so absurd right now is because manufacturers across the board are doing everything to prevent that crash.

They all collectively chose NOT to increase supply with increased demand. So if the bubble pops, they just go back to previous prices without oversupply driving the prices to rock bottom.

The article observes that the cost of frontier intelligence from 2025 has fallen 100x in the last year. It also notes that the energy to run models is also collapsing. Consumer hardware is borked right now because these new algorithms are revolutionizing the utility of a computer. Computing is technology who's cost has been collapsing for 90 years, and its a safe prediction that it will decrease again.

Seems like 3-6 years is a pretty conservative timeline to me. Memory and inference chip production could ramp up massively in that amount of time.

It's true that Jevon's paradox doesn't always apply, although this does seem like a classic case.

But yes, if sold for a negative margin Jevon eventually stops because the decreasing supply will drive up prices.

things are made that will sell for less than the cost of construction

Price is set at the marginal cost. Capital costs aren't in marginal costs.

You'll need a better counter-example than UK railways which suffered from Parliament price-fixing.

I agree, but I would say what the parent is saying is more akin to Say's law: https://en.wikipedia.org/wiki/Supply_creates_its_own_demand

Jev, ASI, RLHF, Jev, Jev, water usage

Jevon's paradox only applies to products with near infinite demand. Energy being the most famous example. I dont think its difficult to argue that compute/intelligence is also a base input into the economy and theres almost no limit to the amount of intelligence the world will want.

AI is not a bubble

When people say "AI is a bubble", they mean economically as a whole, which includes data centers.

Perhaps we need better terminology for "product useful; numbers nonsensical"

I disagree. I own over 1TB of vram at home. I can tell you that it's not a bubble. From my builds, I would rather have cloud, cloud is easier. From running small models like Qwen3.8-27B to large models like Qwen3.8-2.4T. I can tell you that small models will never be enough or match up. Everyone will want the smartest model, not just a good enough model.

NVIDIA will still boom

I think Nvidia is under the same pressure as Anthropic/OpenAI. Nvidia will dominate research and probably keep dominating training, but the real volume is in inference. And for inference Nvidia's lead is only a few months, similar to the lead frontier labs have over open source. Nvidia will sell a lot of Rubin CPX's, but their margin on that will be a lot smaller than B200 because there is so much more competition in that space.

for the first chart (sourced from https://epoch.ai/data/machine-learning-hardware?view=graph&y...), what is the audience supposed to think about that trend line? there's a step after you slap a regression on some points where you evaluate whether there's a real trend or noise, right? i don't see that either in the article or the linked source.

No it'll be fine as long as you do your part and not drive a car, or have AC, or eat meat, or have children, or live in detached housing, or...

I think this is a case where just drawing a "line goes up" extrapolation is incredibly misleading because there is _tremendous_ economic pressure to get costs down, and costs are very tightly tied to energy use. All of these systems are incredibly inefficient right now and have a lot of room to go down in energy use. I'd guess that the absolute _floor_ is burning model weights directly to silicon and that's like a 90+% reduction in energy use.

DS wouldn't be 10x cheaper than the subsidized subscription plans from openai/anthropic. Although it is of course much cheaper than the enterprier/API pricing- I think if you're on the subscription plans, you can't beat that on performance per price.

EBITDA might make sense for the resellers who package up open weight models and sell inference. It is not appropriate for the labs who have billions in debt for RAM, new data centers, gobbling up competitors, etc.

Those real debt obligations are going to want to be paid back.

Who is the "they" that are turning profits?

Definitely. The question is: Is it enough to recoup the enormous capital costs and justify the level of investment they've received. I think there's a decent chance that it will be. But maybe not. And the longer they keep focusing on training new models more so than on inference, the more uncertain I become that it's all going to work out.

Labs are playing money games with EBITDA, which is not uncommon, but also hides the extent to which they are in the red (deeply, deeply, in the red, and projected by them to get worse).

Maybe... I'm not enough of an expert on the financials to say, but it seems to me that inference should be able to recoup the cost of SOTA infrastructure, unless you then also use a large portion of that infrastructure to train new models. And I also think the race to remain SOTA itself is also largely a function of training, because my understanding is that training benefits more from the leading edge of hardware.

But yeah, I definitely don't have high confidence in any of this!

A phrase comes to mind: "Your margin is my opportunity."

The problem source is that the "cost" of tokens are taken at face value from business that are losing money at record speeds. E.g. https://artificialanalysis.ai says "doing task A costed us $10 using OpenAI", and that is the "cost" the OP used as basis for "tokens are cheap". Meanwhile OpenAI is losing $19 for each $1 in revenue... So right now OpenAI should be charging around $200 to do task A just to break even, but that would mean their use base would collapse.

Grep is trivially parallelizable if you care to do it though.

https://iepathos.github.io/ripgrep/performance/#work-stealin...

It might never beat out grep, but it could beat some more expensive to call tools, similar to how heuristics will often be faster than exact answers. Rust Analyzer can be slow at times, I could see an AI tool taking over a subset of its work.

LLM is spicy memoizing, so it can potentially be faster than a tool call. But people will spend a month tweaking and testing to ensure they have the level of determinism they need, which means it's more expensive, and that they should have used actual memoization in the first place.

Even in directed processes the low hanging fruit is harvested first.

Then you have to climb to another branch to get more fruit. The biggest issue with most problem space discovery is you're doing it blindfolded.

LLMs plateau in ability, in which they will start getting ASIC'd.

They don't need to plateu for that to happen. There are companies already building AI on ASIC, and IIRC they were approach 12 months lead time. A 12 months old frontier model (Sonnet 4.5, GPT-5, Kimi K2) for 1% of the price is still a rather good value proposition.

To be clear, I agree with the overall premise of the article!

But I would probably take a long horizon bet that the grep implementation on my machine will remain cheaper than an equivalent ai task, even though I think those ai tasks will become far cheaper over time.

I just think the original comment's model of asymptotic approach is probably more likely to be accurate than the model of the line blowing through this grep-like cost level.

Yeah it's a pretty poorly specified problem. It needs to be some kind of "equivalent task", but it's not clear how to define that.

calling an LLM will soon be cheaper than a grep

From a computational standpoint this is obviously nonsense, but from an attentional one I'm not so sure. It may already be more attentionally expensive to use grep in some cases, such the moment you need to remember a non standard arg. And if this applies for performing a simple http operations, then it certainly applies going up the complexity chain.

AY-3-8913?

Barring a true breakthrough in something like quantum computing or room-temperature superconductors

Won't that also help grep and then move the asymptote down more?

The author observes that a call to GPT-5.6 Luna is only 4-5 orders of magnitude more expensive than grep, and then predicts that at current rates of progress, calling an LLM will soon be cheaper than a grep.

At some future point where LLM hardware is cheaper than simply running grep, then grep equivalent would benefit from those selfsame hardware improvements and be cheaper to run as well, probably still by the same ratio.

There seems to be a mistake in the cost comparison between 2025 and 2026. The 2025 chart axis is the cost to run the entire "intelligence index", and the 2026 version is a weighted average cost per task.

I don't disagree with the thesis here, I just don't think costs are coming down quite that quickly.

So this week's coordinated shock and awe campaign is about "open models" and "prices are falling rapidly". Also on the front page:

https://news.ycombinator.com/item?id=49815526

GPU case seems very weak. The graph is impossible for me to reason about at least. You could draw basically any trend line through that GPU graph and it would look equally plausible to me. The main takeaway I get is that the NVidia H100 from four whole years ago is barely different in efficiency from the state of the art, which is surprising to me, and seems to indicate the exact opposite of what the article says.

A much deeper analysis on the falling price per task was published yesterday by Epoch AI [1]. It's a real statistical analysis and comes to more defensible and grounded conclusions. The headline takeaway is:

The cost of a given level of performance often falls fastest right after that level is first achieved, that is, when it is state of the art (SOTA). We see this pattern on three of our five main benchmarks of AI capability. Averaging across all five, cost falls 66% per quarter (75× per year) for performance that has just debuted as SOTA. Two years later, prices fall half as fast, at 32% per quarter (4.7× per year).

but the analysis itself has more nuance and is a quite interesting read.

[1] https://epoch.ai/publications/the-plunging-price-of-thought

I strongly agree. We _will_ get there with solar, wind, and batteries.

Eh, it looks like micro reactors are coming on line much faster these days.

Nuclear was never going to give us too cheap to meter regardless of politics. Uranium just isn’t that cheap. Fuel costs are lower then coal or gas, but not so low that operators would just not bother to charge for it

It's a deliberate reference/meme that is basically used to acknowledge the precedent of overly exuberant predictions of cost in an emerging technology but argue "however, this time it's true".

Of course, perilous territory for future irony depending on how your prediction plays out.

On the other hand, this did work out in other areas. I pay a flat monthly rate for all-I-care-to-eat internet access, for example. My email provider has limits on storage space but I don’t get charged per email sent or received. It’s not a crazy concept on its face, nuclear power just didn’t work out as well as was hoped.

your repricing has nothing to do with subsidising which means selling at a loss. Within this year, the real prices have gone down more than 10x on average which is way more than the teeny 25% you are fighting for.

where? I mean the API costs

Not really related to the central point, by but I couldn't help but get caught up by

Generally, models intended to be run locally will be much smaller, such as Muse Glimmer or Qwen3 Coder.

That is such an interesting set of models to use as examples here. One being essentially obsolete on release a month ago, and the other being completely ancient in LLM time. I really wonder how they landed on those two.

It's true that LLMs "want" to be be local, but they won't shift broadly to being local until there's a sufficiently large supply of VRAM or (at least) "unified" memory from the manufacturers. (I'm also assuming here that radical regulatory changes like government bans of local models aren't going to happen.) So (AFAICS—I am no expert) the future of LLMs over the next few years comes down primarily to the nitty-gritty of how much memory fab capacity will be added and when, and to a lesser extent of what happens to future demand from LLM SaaS services (& maybe their existing stock of hardware if they get in trouble). (I'm also assuming no roughly-AGI-sized leap forward which makes the frontier models of the near future vastly more valuable than the near-fontier models of today.) For the incumbent manufacturers the high-margin business is selling to LLM SaaS providers who use VRAM efficiently, but the high-volume business is getting chips into millions of laptops which will use VRAM very inefficiently. I assume that they will want to move from high margins to high volumes as they build they physical capacity to ship higher volumes, but they seem to prefer to do it at a stately pace. Hopefully some jostling from Chinese competitors, and maybe a dropoff in demand from data centres, will speed things along.

I agree with OP that we will continue to see improvements, but there are also some serious bottlenecks ahead of us:

- Energy is not infinite, neither energy efficiency is. - Datacentres neither. - Benchmarks are an abstraction of real world problems!

On top, there is an overall "economic" aspect that most of the people miss: every change carries a certain degree of risk (lose money, reputation, customers, death of people, ecc) that very few want to take and a lot of changes(e.g. rewrite some piece of SW in another Lang) don't produce a positive economic impact.

We are likely to see LLMs integrated into every part of computing as infrastructure, not just as a product, in the next year or two.

Of course, for collecting better telemetry using local AI for analyzing video from camera and audio from a microphone.

jgrep is not meant to replace grep. It’s supposed to extend it to new use cases. Keyword search will always have a place!

Improvements that affect local AI - Mamba...

I just stopped reading at that, for anyone else, Please find a better source and take everything in here with a grain of salt.

IMO

The number one improvement that mattered for local AI was llama.cpp, partial offloading to system cpu/ram. The next was quants, being able to take fp16 and turn it to q8, q4 etc. The next IMHO is unsloth dynamic quant, that have been able to do mixed precision so we have UDq1/q2 that is actually pretty damn coherent. Allowing individuals to drive K3 locally even if it's at Q1/Q2. Then MoE changed everything for everyone, cloud and local. The other is integrated GPU, Apple, Strix Halo, DGX Spark. Then all the extra improvements like MTP, DSpark, etc. Of course there's many other additional things that have mattered too