A shame that this should if anything, lead to consumer DRAM prices getting even worse.

Why would more RAM supply lead to higher consumer prices?

Depends on what proportion of Samsung's output is current HBM4 and HBM4E DRAM.

Everything in their statement can be true and it be a bad thing for consumers of non-HBM RAM.

"HBM capacity to expand to 250,000 wafers a month"

So let's say their current HBM capacity is 100k wafers/month (pure speculation/random number for illustration), and their total RAM capacity (including HBM( is 300k wafers/month, then non-HBM capacity reduces from 200k to 50k.

Keep in mind that total manufacturing capacity is increasing as well. Perhaps not enough to fully offset, but it's incorrect to assume that supply capacity is flat.

HBM4 and HBM4E DRAM are NOT the DDR4/5 that consumer markets need. Capacity allocation is leaning more to data center grade HBMs so less to produce dedicated DDR4/5 DRAMs. Supply demand will further drive up the consumer DRAM price! Note, the article mentions NO of new fabs is being constructed (all semiconductor manufacturers know that constructing more fabs means the boom/burst cycle will eventually kill them, so no one create more fabs) Perhaps the federal government need to step in here - the market doesn't fit the issue.

Plus it takes 3x the wafer capacity for hbm than the same byte capacity in dram, so we’ll likely see the consumer market be decimated here

The consumer market will not be decimated. Memory of all kinds will get very cheap quite soon because of supply and demand and substitution.

Might lower the prices of Mac Minis, Studios etc though

No, completely wrong. Supply and demand will definitely fix this.

More HBM4 will drive the price of it down. That will make it less profitable to produce. That will mean firms shift some production back to more profitable "consumer" RAM. Increased supply will lead to lower prices.

A shortage is always followed by a glut.

That will mean firms shift some production back to more profitable "consumer" RAM. Increased supply will lead to lower prices.

Long-term, sure. But the article we are discussing is about next year, non-HBM supply going from 60% of their output to 20% (HBM 40%→80%).

What specifically did that say that you think is wrong?

Supply is currently moving the wrong way. Supply and demand for consumer DRAM will get even worse, even if it will "definitely fix this" in the long run.

And the fact that production is moving the wrong way suggests that in the short term supply and demand is actually doing more harm than good. HBM is so profitable it's radioactive.

I don't know where you got the idea in another comment that all memory will get "very cheap quite soon". They're not increasing total memory production. (much/yet)

What specifically did that say that you think is wrong?

You said:

the market doesn't fit the issue.

I presume you meant "fix" not "fit". You are wrong that the market will not fix the issue. There are literally millions of observed cases across many markets of increased demand (or decreased supply) leading to higher prices leading to increased supply leading to lower prices, which is why it's a fundamental law of economics.

The price of RAM will fall.

I feel like everyone here forgets that the DRAM industry is operated by one big cartel. The rules of supply and demand don't apply here. Now that prices are up, they will keep it up, cutting supply as needed in order to do so. Every big DRAM manufacturer in the industry has been charged with price fixing, among other things: https://en.wikipedia.org/wiki/DRAM_industry_price_fixing

There are so few players in this space they can coordinate and collude pretty easily.

China is about to massively compete. And doesn't Micron and someone else have new fabs in NY coming online ? I can't imagine they're going to exclusively make HBM.

Ram always goes through these boom and bust cycles. They're going to increase supply but not so much that they're eventually left with huge overproduction as usual.

Of course if Google researchers come up with more ram reduction tricks for newer AI architectures that could make a huge difference as well

Part of the implication is that factories that could be producing consumer-facing DRAM like DDR5 would be retooled to produce HBM instead, leading to even less total consumer RAM production.

Will that be enough for AI's hunger?

It will be just in time for when AI will run very well on consumer hardware and the need for data centers will collapse.

Or even more amusing in time for the bubble to burst, I hope the big RAM makers end up holding the whole bag for that. Greedy bastards.

Since you’re not greedy when there is a memory glut I’m sure you’ll be willing to pay extra to make it fair.

Not at all, I’m going to relax with a cooling drink and watch the consequences of this unbelievably destructive and wasteful venture implode. I also adore the idea that it's on the customer to be "fair" to a company that dumped us a group in favor of chasing B2B money.

Their choice, their consequences.

Internet is a fad, it will implode at any moment.

In fact as a business opportunity it did just that because the massive investment in the mid-late 1990's was without anything like a connection to profitability or sustainable demand. It took years after that crash for the concept of e-commerce to begin both a recovery and evolution into what we see today.

I expect something similar for AI. It's useful tech... just not very profitable tech, and certainly not to the tune of trillions of dollars worth of public demand. The promises of superintelligence, replacing everyone, and the rest of the hype will die with the companies who made the promises, but the tech will survive and thrive.

The promises of superintelligence, replacing everyone, and the rest of the hype will die with the companies who made the promises

The hype will die but there are many reasons why super intelligence and robots are a separate entity from said hype.

Look, back a few decades and tell someone our gdp would be in the trillions and it's likely they'd have a hard time believing you. A huge portion of our products that we use day to day would be complete science fiction to them.

All we're negotiating at this point is the timescale it will take.

Nearly the entire reason we're in this mess is because RAM/SSD/HDD makers are terrified of AI bubble bursting.

Long-term contracts, not building new fabs - they're in full on hedging mode right now.

Seemingly the AI bulls are either not sufficiently optimistic, or not sufficiently wealthy(?!), to build new fabs as joint ventures with the manufacturers in which they agree to assume most of the downside risk?

That's never going to happen. By the time you can run current frontier models on your $10k desktop the frontier will have massively advanced and people will want those models instead.

This + your ROI in $10k device will be always lower than busy datacenter, its literally math. They sell free compute to others when you dont use it, you will never sell at that level or even you magically sell home compute, you will not compete at price

I'm not sure if this prediction will hold true.

We're not seeing the progress in those "frontier models" that we have previously seen. There's certainly still gas left in tank tank, but we're way into the diminishing returns by now.

Cloud inference still beats hardware investments by orders of magnitude of course, but that's only if your data doesn't really matter to you.

We are certainly not in the diminishing returns phase for LLM progress. No sign of that yet.

I’ll grant that for specialized applications like coding agents and mathematics, but even there I suspect that most the real gains are actually taking place in the harness.

But I suspect returns may have already diminished into negative territory for at least some other use cases. One of my least favorite job responsibilities in this brave new era is figuring out how to avoid performance and behavior regressions when an older model were using for some application reaches end of life. It’s getting uncommon for me to look at our benchmark results and say, “Oh, good, it does better on one of the newer models!”

suspect that most the real gains are actually taking place in the harness.

Part of the reason harnesses work well is you can run a lot of agents in parallel. That doesn't slow down demand.

That is true, but the eventual realization that more machines doing more coin flips in parallel does not mean "more work gets done" might.

LLMs are amazing tech, but they're terrible without oversight. More agents faster just makes reality collapse on them quicker.

But yeah, you're right, temporarily, this will still push demand. But the topic was about "diminishing returns" as in "tech getting better". Not as in "customer spending".

I had actually been thinking more about all the non-LLM functionality that go into the harnesses. I'm not going to name names and I haven't done any rigorous testing, but my general impression is that choice of harness matters more than choice of model. In terms of basic task completion success specifically, not code aesthetics.

Google scholar has a flood of papers showing LLM diminishing returns on pretty much every facet

https://scholar.google.com/scholar?hl=en&as_sdt=0%2C23&q=llm...

they really dont want to hear this bro lol

It’s a constant tension in computing that has been around since mainframes and clients… Neither is going to disappear. My general feeling is normal people care more about how thin and light something is than their privacy, so if data center powered LLMs will have a strong future.

I’m not so sure about that. Already AI vendors are back to cutting prices to try and keep customers from cutting back on their usage. My own employer is working hard at pivoting to much smaller fine-tuned models for established use cases, and seeing model performance improvement in addition to large inference cost reductions. Being able to run them locally hasn’t exactly been a disaster for devex, either.

It may turn out that demand for SOTA frontier models isn’t so limitless after all.

Every product follows demand curves. At a price of 0 you could find infinite usage. This has nearly zero relation to how much it costs to provide the product.

Except of course it relates. All else being equal, we will prefer $X COGS over $2X COGS because that helps us with both profit margins and price competition.

It relates in the sense there's a minimum cost of production without losses, not the actual price people are willing to pay.

It's going to happen very soon, which is why these frontier labs are scrambling to shut down open source language models. There's an existential risk threatening their obscene returns.

It's going to happen very soon

Why? You can't just assert it. There are very good reasons to think it won't happen soon, and you've given no reasons to think it will happen soon.

You asserted that it was never going to happen first

There will be demand for both

What if the AI companies start selling the hardware with their models?

I.e., you get a locked down device with access to their AI and maybe a way to run third party apps. Of course the developers of those apps will have to pay a percentage of their revenue.

Sound familiar?

More likely they would sell a chip with the model etched in silicon for the dramatically higher token/s.

Most likely those models would be extracted from the hardware in no time.

About 2 years ago I bought some SDRAM or something like that, DDR4 or DDR5, don't recall offhand. A few months ago I looked at the price today, and it was over 3x as high. That's just insane. Governments need to do something against this abuse system that AI amplified here.

What abuse? Responding to supply and demand fluctuations isn't abuse, it's the proper operation of the market.

Would if one industry gobbles up an important resource to the point that other consumers lose practical access to it, shouldn't there be limits?

If others are willing to pay more, that a signal that more value is created if the resource goes to them rather than to the more price sensitive customers.

This logic falls apart completely when gamblers allocate one trillion to bidding up the price, denying access to the resource to people who are responsibly spending their own money. The ideal world of the imaginary perfect free market never takes into consideration the messiness of the real world, like the fact that people will spend extreme sums of money irrationally. It can take years for this effect to correct, damaging the market severely in the meantime.

Alternatively, the money invested can be rational because it prices out competitors and establishes a monopoly, after which point the monopolist earns their absurd investment back with complete control of the market. This is also bad.

This has zero to do with the company that is making the product and all to do with the company buying them.

The problem you have here is hundreds of different companies are doing the "gambling" in a non collusionary manner so it's going to take decades in court to prove it. Are you saying governments should do an authoritarian take over of RAM allotment?

Where is the additional supply resulting from the sustained increase in demand?

Check the article we're commenting under

Governments need to do something against this abuse system that AI amplified here.

Which governments, and what exactly would you want these governments to do? The demand is global. There are no levers a single government can pull to meaningfully influence the global demand without fully committing to an protectionist economic policy, in which case the U.S. doesn't have the facilities to magically pop up world class fabs overnight, and South Korea and Taiwan don't have the market demand that the U.S. generates to justify their investments in making these chips and China lacks the IP to be able to build anything comparable to Nvidia's silicon at the moment.

No one has all the cards and no one controls all the levers.

Governments have sued DRAM companies for price fixing and forming cartels before: https://en.wikipedia.org/wiki/DRAM_industry_price_fixing. They can absolutely step in and if there are evidence of price fixing or anti-competitive behaviours, can impose fines. The choice of whether to do that though is political.

Is there any allegations of price fixing in this case? The demand just skyrocketed. Other than legislation like the defense act in the US I don’t know of any governments that have jurisdiction over these ram makers that mandate companies produce more of a product.

Especially not for consumer goods.

If they're charging one customer $X and another customer $5X for the same chips... there's got to be some law about this.

I missed that this is happening, is it really? I thought it's 5X for everyone

Supposedly OpenAI is paying 2025 prices.

No one has all the cards and no one controls all the levers.

I'm as fanatical a free-market fundamentalist as you'll ever meet, but if someone were to argue that government has a role in preventing bullshit like OpenAI's unilateral 40% attack on the entire DRAM market, backed by nothing but funny money, I would have a hard time coming up with defensible counterarguments.

Your attempts to insult what they are doing doesn't show they did anything wrong, but it does impeach the validity of your own argument.

Hit dog hollers, sounds like.

That's abuser logic.

How much memory does it take for the consumer market not to be totally screwed?

Can that share of production be allocated to consumers, and the AI fights over the rest?

3x? I bought 128GB DDR5 SODIMMs for 300 eur. Now it's 4000 eur.

Damn you were lucky. I bought 96GB DDR5 last year for £600 UKP. They're now retailing for three times that.

Bastards.

What's the main blocker (other than the current inflated cost) for using HBM instead of DRAM as the primary memory for consumer electronics?

Nothing except CPU manufacturer choices. Mac laptops use it and they're consumer products.

People will have to get used to buying a fixed amount of RAM with their CPU but thats unlikely to be a problem.

The normal non-tech-savvy person already does this. They simply don't know that their ram is upgradeable or something else breaks first, before having to touch ram.

MacBooks use regular soldered lpddr5(x) RAM. Same RAM as every other laptop manufacturer, they just use more lanes to achieve a higher bandwidth.

Perhaps more noteworthy for general home and business computing, doesn’t it also allow for lower latency?

More channels or soldered memory? Channels are basically RAID 0 so it depends what you're measuring. Soldering memory down was the only way to use LPDDR5X so if you wanted the best memory you had to solder it down. LPCAMM2 exists though so newer devices can use that instead of soldering them down, but not all devices would be able to fit the required LPCAMM2 slots.

SOCAMM2 allows for removable ram in nearly 0 added space.

That's tiny! But it still depends a lot on form factor. LPDDR5X is used in phones and making memory removable would have its compromises. You may even have compromises in laptops. Look at how tiny a MacBook Air's mainboard is and you'll see the RAM modules on the same package as the SoC. SOCAMM2 is too large for that but a variant with only two modules could possibly work.

Apple's "unified memory" marketing is so strong that even tech literate people seem to have this misconception!

They have managed to pull this sort of thing off many many times. https://en.wikipedia.org/wiki/Reality_distortion_field

Yup, the only reason Macs have higher memory bandwidth is because they use more memory channels, which gives them a wider bus. Both Intel and AMD only allow more than dual channel memory on server class processors these days.

The “Apple only does X better because they do Y” thing has been a meme for ages. I remember dismissals like “PowerPC is only faster at math because it has more integer units” or something along those lines, and thinking, uh, isn’t that a good thing?

I am just saying it's not magic and x86 is capable of doing the same. Quad channel memory used to be more common in consumer hardware but now it looks like you don't even have the option anymore for desktops. AMD's Strix Halo was the first sign to reversing that (it has quad channel memory!) and hopefully we see more of that in the future.

AMD does market segmentation and limits consumer chips to dual-channel. You need to cough up the dough and get Threadripper for quad-channel. Or even more for Threadripper Pro to get octa-channel.

Afaik steam deck is quad channel, despite it being a pretty low end chip (with a decent GPU though)

There's a qualitative difference between "they're doing a different thing" and "they're doing the same thing, tuned differently". GP is saying that this is a case of "they just tuned it differently".

This distinction doesn't change what the performance numbers look like today, but it does inform what changes would be necessary for those numbers to look different tomorrow. E.g. Apple Silicon isn't fundamentally orders of magnitude more efficient than x86, they just used smaller features. Newer Intel and AMD chips made on equivalent processes _also_ get similar efficiency gains.

The point is that "unified architecture" is a buzzword unrelated to what actually makes things fast.

The memory bandwidth is a small thing compared to the massive win you get by not having to move data between two memory pools at all.

Often you don't move memory around as a programmer, but that's exactly what happens in the background.

It's the address space that's unified, not always the physical hardware.

The data movement (when needed) is handled transparently in the background by page faults and other tricks.

Threadripper have that extra bandwidth and M5 still is faster

Can you link a specific benchmark?

Keep in mind that non-Pro threadripper is still only 256 bits wide and Pro is 512. And the memory is 30% slower than with an M5. So an M5 Ultra has 3x the memory bandwidth of the best threadripper.

Don't some AMD full server CPUs have 12 memory channels ?

Having unified memory is a real advantage though, it's not a reality distortion.

Agreed. That's why it's a good thing all computers made in the past 15 years have unified memory, not just macs.

https://x.com/Lina_Hoshino/status/1820947147312820497

It isn't really, as long as you don't care about power consumption, physical constraints and money. Basically desktops <2025 (and hopefully >2027).

DDR is optimized for latency and stability at the cost of bandwidth whilst GDDR is optimized for bandwidth at the cost of latency and stability. GDDR is pushed so hard these days that a small percentage of errors is expected and corrected because this is still faster than running it slower but more accurate.

GDDR7 often has 10-20x the total bandwidth but 3x the latency of DDR5. Graphical workloads want as much bandwidth as possible but care relatively little for latency. Conversely, applications love low latency but don't really see any performance benefit from higher bandwidth.

So basicallyt you have workloads that are diametrically opposed and running unified memory forces you to compromise.

It's not that there's a blocker. It's that it takes roughly 3x the manufacturing capacity to produce an HBM package at the same storage capacity as DRAM. We are sacrificing total bytes for bandwidth.

How so? FEOL is pretty much the same, BEOL is almost the same save TSVs, the packaging tech is different and more advanced, but not exactly 1:1 comparable. Do TSVs really occupy 3x the area of DDR IO's?

See remark on the slide 11: https://www.servethehome.com/micron-evolving-memory-architec... That presentation is by Micron.

direct link: https://www.servethehome.com/micron-evolving-memory-architec...

Yuck. Time to build an xSPI/HyperRAM workstation (if only these had multi-bank chips).

Sadly the $ per byte of xSPI and HyperRAM quite high

It doesn't really say that it needs to use 3x more area, but that 3x more gets consumed due to "advanced packaging and manfuacturing complexity". Which doesn't properly explain why it consumes 3x more and could simply mean they have a bad yield and 2/3 produced is garbage.

I don't fully understand the source of the "total bytes" constraint, but a major factor may be because HBM4 / HBM4E can only make use of the footprint directly above the processor/logic die (or in direct vicinity of its interconnect), while traditional DRAM can be placed further away where there's lots of real estate on the motherboard.

I gather a practical max ceiling today is a stack of 16 chips in height yielding 64GB?

These chips have a massive bus size of 2048 bits, instead of the 64 or 128 bits (dual channel) used by DDR5. That's what gives them their order-of-magnitude bandwidth speedup. But even though they technically pack in more capacity per square millimeter of motherboard, I gather they take up more space than older technologies once you account for the vias and interconnects to route all those signals.

Thanks for that, just went down an interesting rabbit hole. Many of us were hoping this re-tooling would eventually trickle some fast RAM down to DRAM-exhausted PCs, but given it would require a rearchitecture of the motherboard it's unlikely.

HBM4 has over 2048 signals to the processor’s PHY with tight signal integrity requirements that require the HBM stack to be < 0.5 mm from the processor die. That’s why HBM integration is done with interposers (soldered on the package). So, it’d be the CPU package that integrates it. Motherboard is too far away.

I think people might prefer the lower idle power consumption from lpddr over the better bandwidth in hbm in battery powered stuff. That said right now the price is definitely preventing us from finding out.

Idle yes, but HBM energy consumption / memory operations seems to be a bit better than DRAM.

Only if you’re running at 100%. Consumers generally do not.

That hasn't been true since early HBM2 days, before the controllers standardized on power/voltage management and did things like leave them in P0 to ship on time

LPDDR/DDR/GDDR generally still win over HBM when there’s no data being transferred. HBM is primarily better in watts/byte transferred. Consumer electronics spend most of their time idle.

Racing to idle is a very common power optimization technique

Right, but HBM idle is significantly worse than LPDDR idle or even DDR idle for that matter. That matters a lot precisely because the device is idle most of the time. Your idle power draw dominates.

> What's the main blocker (other than the current inflated cost) for using HBM instead of DRAM as the primary memory for consumer electronics?

HBM is meant to be integrated into the same package as the CPU, so no more DIMM sockets. It also has higher latency apparently.

HBM is a stack of DRAMs, so there is no “instead”.

The vias to enable stacking is a significant amount of the die area.

If they're talking about production capacity, that is some product of die area and process steps, right? It doesn't have to be 3x die area, just 3x lower factory throughput for the same number of functioning memory bits.

Why would you want/need to?

The advantage of HBM over regular non-stacked DRAM is memory bandwidth, which also requires a super-wide memory bus - 2048 bits wide for HBM4. Compare that to the 128 bit wide bus of a modern CPU.

So to take advantage of it on the desktop, or anywhere else, you need that 2048 bit wide bus, and a processor capable of consuming 2-3 TB of data per second!

These are not normal requirements, other than for a GPU.

SIMD (and especially the modern matrix extensions) can use as much bandwidth you can throw at it.

Without further clarification, that statement seems impossible. "As much" being unbounded and all. You should expand what you mean.

This will blow your mind, but it actually is pretty close to being unbounded. :)

Consider the 'MMA N matrices' primitive modern CPUs are starting to support. For the current generation of CPUs, N is a constant like 16 or 32, but there's nothing preventing it from being 1024 or larger if we have more memory bandwidth.

All this with a single instruction.

Surely something prevents it being 1 quadrillion bits per instruction? Since that’s well within “unbounded”.

Nah his point is a bit simpler. Vector units can process a limited amount of data per time unit. Memory can load a limited amount of data per time unit.

If you have infinite memory bandwidth you just move the bottleneck back to compute so both have to grow simultaneously in lockstep.

What you should have said is that CPUs have so much compute headroom for matrix vector multiplication that simply adding more memory bandwidth would make them faster so every improvement in memory bandwidth is welcome.

Right but if HBM memory is all that people want to produce, then building a CPU which can use it use it would be useful on it's own merits.

But in reality we also already have unified memory architecture systems, integrated graphics etc.

People want to produce hbm because it’s more expensive and more profitable than regular memory.

There is a world where scale and experience means it's about the same though, is the thing.

And memory is already expensive. It's downright hard to even get it though - you frequently would prefer not what's cheapest, but whatever is in largest scale production.

Scale and experience almost entirely share between HBM and normal memory. And they're both in large-enough scale production to not have a big difference on availability; if you're willing to pay HBM prices you should find even more sellers of DDR.

The only way I see HBM becoming competitive for consumer CPUs is if they solve the yield issues. Or if AI crashes so hard that people are putting those GPUs on fire sale and salvaging mass quantities of HBM off of them.

I was thinking this, but I doubt that the HBM is so easy to repurpose.

I’m assuming that it’s mounted in the same unit as the GPU, not in a handy-dandy socket.

It could be cracked open and extracted no doubt but if it’s glued in that’s going to be nigh on impossible to extract.

I had been pinning my hopes on HBM coming onto the second hand markets after there three years or so of use, perhaps I’m wrong.

Now make some DDR5.

HBM is DDR5 in a different package.

The beginning of the bubble pop?

how would you even derive that conclusion from the headline/article?

https://youtu.be/k_gaZjXD5OY?t=200

On a related note, I was reading yesterday that apparently the real bottleneck for Chinese production of AI accelerators is HBM production, not processors or ASML equipment.

The lack of ASML EUV machines certainly hurts, and pushing DUV so hard results in abysmal yields of good chips, but you can compensate by running more wafers or making smaller chips, and the net result is that Huawei's Ascend production volume is limited by CXMT's HBM capacity not processor dies.

The problem is that HBM manufacture requires many steps (die thinning, via drilling, plating, alignment) where the equipment used by everyone else (Samsung, SK Hynix, Micron) is also blocked by sanctions, so the Chinese are having to develop all of this themselves too, which they have, but yields are currently low, even when using shorter HBM stacks.

Tokens per second is almost entirely memory bandwidth at inference time, training obviously needs more compute but you can add more chips for that.

Not quite, it's got quite a bit more complicated with agentic use cases.

Prefill (input tokens) is heavily compute bound. And the ratio of input to output continues to rise, as typically in agentic sessions you have a few tokens output for a tool call and (many) thousands of input from the tool result.

Then you have cached input tokens, which is a totally different issue, system RAM or NVMe bound.

Obviously output tokens is VRAM memory bandwidth bound, but this is less and less of the bottleneck these days for overall agentic speed.

I read HBM yields are 25-30% (vs 80-90%) making them 3 to 5 times as expensive. They are 4-5 years behind, that probably means 1-2 in Chinese time.

China should invest in an analog inference chip. It's a hail mary but why not.

That would be like skipping land line phones for mobile...

Developing countries have done exactly that in many cases.

The two challenges there are firstly - that analog design has been a separate electrical engineering school for most of a century, so there are few who could design it - and secondly - that every single chip will have subtle variations in its computations, necessitating some sort of model finetuning per chip. Possibly the chip could be characterised at the factory, and ship with the characterisation data burned into a controller rom or something, but if that doesn’t pan out the whole thing is likely a non-starter.

If it could be made to work, you could run a fable-grade model in tens of watts.

Or we treat them like humans and make each one do job interviews to work out what tasks they’re suited for.

The only thing the West is achieving here is that sooner or later China will be able to compete on their own terms rather than ours. It may buy some time but the end result is very predictable.

It baffles me how shortsighted the policymaking here is. Like, what did they expect to happen?

They probably weren't expecting AI foundation models / model research to be quite as fungible as they are

Yes.

Over the last 20 years the economy has become dysfunctional. It no longer really resembles a free market; monopolies have established barriers to entry everywhere. And the biggest investors are awash with helicopter money that's been doled out for favors by the political class.

So those investors have tons of cash to burn and surprisingly few opportunities. Even within Silicon Valley/VC there are surprisingly few who seem to really understand the fundamental economics of software. Or perhaps those economics just aren't that important when you have billions of dollars on hand and cash is obviously not going to get you a return. Any whisper of possible exponential growth is worth throwing money at. Crypto? Why not. AI? Why not. Datacenters? Why not. Tulips? Why not.

This is by all definitions an empire in decline. Everything is broken or fake. Everyone is afraid to do what needs to be done. Power forbids it. So we're all just waiting for the other shoe to drop. Our secret police aren't as bad as the late stage USSR's yet, but hold Uncle Sam's beer...

(That's not a recommendation to try and time the nadir, by the way, as it could easily be 50 years away.)

It started way before the last 20 years, the past 40 years have created the conditions for this to happen.

Overfinancialisation of the economy while removing safeguards to keep markets functional, like anti-trust enforcement, are the main forces behind this erosion. Through finance the focus of companies is completely shifted away from producing good products and services, and into how to extract more paper wealth from existing structures to the detriment of products.

Not enforcing anti-trust and letting behemoths to form which cannot be competed against since with their amount of capital they can either buy their competition outright or just price dump for long enough to make competition non-viable.

It's the failure of neoliberalism, and that agenda has been pushed into Western countries since the 1980s-1990s, it made enormous wealth for the few at the top while eroding whole societies, economically and socially, it's all dysfunctional.

I feel like my point is being over extrapolated a bit. All I was saying is that people in 2020 believed AI companies had much more of a moat then they actually do - that other companies would struggle to obtain enough data (y’know, downloading the internet to train models wasn’t legal back then) or resources to compete

Datacenters and AI obviously work and come with a credible value proposition and path to profitability.

Comparing this to crypto or turnips makes me doubt the merit of any other opinions your comment offers.

The initial growth of the internet was a boom bust cycle, and as you see the internet is still here. This is more about the economic paradigm that is being used to grow a technology versus the economic paradigm the technologies rate of growth can support over time.

Just to give some rough numbers in the past 5 years the amount of GPU compute installed in FLOPS is somewhere over 5 times all the CPU flops that have ever existed.

This has nothing to do with AI being good or bad or being able to produce things and economic value. It is by far the largest and fastest growth of any technology ever and we have zero clue about the economic stability of this grand experiment we're performing.

The internet is a much more appropriate comparison than crypto or tulips, which bring no or negligible value.

Some differences to the market at that time seems to be that during the dotcom bubble, many of the companies had little to no revenue.

Leading AI labs already generate enormous revenue. The investments into capacity are needed to address the current demand.

The situation seems somewhat less speculative.

That said, I cannot predict how AI capabilities will develop and how demand will respond.

Should capabilities plateau hard and soon, maybe the demand will not be there for the compute investments.

If it does not, and instead AI applications in robotics, science, and self driving expand, chances seem reasonably good that the demand will be there, no?

And as for the economic stability, much of the investment comes from existing giants like Microsoft, Alphabet, Amazon, and Meta, who have the necessary cash flow.

These companies are less likely to collapse than some of the ones during the dotcom bubble.

if you sell them ASML equipment your market gets destroyed tomorrow instead of 10 years later so what options do they have?

They always had the option to do that

As other comments pointed out, that was always China's plan, the value here is really that when the Chinese overlord annillate the world's industrial competition, they'll have the moral high ground of being out of self-reliance and hostility by western monopolists.

Edit:

And, that, my fellow keyboard high-minded thinker but powerless corp peons, was also part of the China's plan all along.

Was it also the Chinese plan all along to annihilate their own birh rate?

Was the Great Leap Forward part of the plan too?

Was it also the Chinese plan all along to annihilate their own birh rate?

No, that's generally what happens when countries stop being poor. Taiwan, Singapore, Korea, all have similar fertility rates.

But none of them had a government-mandated one-child policy (maximum, not minimum) for over three decades. That's a pretty major difference between China and neighboring countries.

And yet all of them have an even lower fertility rate than China.

On the one hand, you're proving my point. On the other hand, if you think that China's headed for a demographic disaster, so are all of those countries, and your thesis of blaming the CCP for everything... Falls a bit flat.

China is headed for a demographic disaster, though. High schools are incredibly packed, middle schools, not so much and primary schools and kindergartens are kind of empty.

Yes, they literally named it the ‘one child policy’. It was just overly successful.

The official one child policy didn't actually have much of an impact. Birth rates had already fallen a lot, despite the official "lots-a-children" policy.

The one child policy is gone now but birth rates haven't really moved upwards.

Of course it had an impact, if nothing else, a cultural one. It basically eradicated the concept of large families.

Eh. Industrial policy in a big country is always the output of politics between stakeholders, anthropomorphizing it to a "guy with a plan", especially an "overlord" looking to "annihilate" is probably getting it wrong.

A friend of a friend is married to a member of the national academy of sciences in China. I heard at dinner a few years ago that he had been lobbying for more onshoring of chip production for years before the Nvidia restrictions, and always heard "you're right, maybe next year we'll be able to fund that". Then the restrictions came in and everything changed.

There are literal 5 year plans, eh?

And they are more about sounding good than about reality. They also affect a much smaller share of the economy than they used to -- which is very good! The original five year plans caused nothing but misery and starvation.

I’m pointing out it’s disenguous to say ‘it’s not like there was a plan’, when there was explicitly a plan with that title on it. Especially if the plan didn’t work out well.

It's disingenuous to characterize "don't anthropomorphize the political process" as saying no planning documents exist as part of that process.

And, to the point I was responding to originally, we can check those documents! They don't speak of "annihilating" other countries' industrial capacities.

[...] that was always China's plan, [...] they'll have the moral high ground

It's a possibility. It is also possible that they will steal yesterday's technology and still pretend to have moral high ground.

I do not wish China and Chinees wrong but most probably communism will work as it always do - by faking thing and bringing self-harm to their own citizens.

In free markets, everyone steals everything they can, and stops everyone else stealing from them as much as they can. This is normal and expected. The ideology says that it will even out and converge to a fair situation.

Well Soviet Union tried to be self-sufficient, but the truth is such strategy loses to global free trade. Whether we want it or not, global trade is what keeps the world from WW3 for now, not the nuclear MAD.

What the West is doing is forcing China into a situation which is impossible to sustain when their economic dividend runs out. Especially considering these investments are fuelled by an extremely high level of debt.

And it’s not a given that they’ll catch up. I think it’s fair to say that their development of jet engines is on track to be obsolete before they’re competitive.

They may in principle have established the technologies to be self sustained but that’s only relevant and sustainable if they develop an economy based on domestic consumption rather than exports. Which they are struggling with as well.

They may in principle have established the technologies to be self sustained but that’s only relevant and sustainable if they develop an economy based on domestic consumption rather than exports. Which they are struggling with as well.

Is this really true? Can't they just keep exporting "forever"? What requires them to find a domestic audience? I can think of some other countries that have done quite well mostly without finding a domestic audience (eg, Switzerland comes to mind, and frankly Europe might largely be in the same bucket).

https://www.imf.org/en/publications/fandd/issues/2023/06/sup...

Totally non-biased of course.

their economic dividend runs out

i dont think it will - they still have a chokehold on battery production (which implies dominance in the EV sector), and they also control the vast majority of all rare earths. The chinese gov't can subsidize chip research/development with these other sectors that are dominating.

The west is the one that need to watch out for their own economic dividend running out from the past century.

The "abysmal" yield using DUV is an overblown statement by western commentators who don't look closer at the development. Certainly yield is worse than with EUV, but after multiple iterations of development, yield has become pretty good, within economically acceptable bounds, still making scaling possible. Volume is still ramping up. A lot of capacity will come online in 2027. The removal of western middlemen such as Mediatek actually improved the economics. Further yield improvements are still coming. Volume is already so high that domestically made phones and NPUs are making a real impact, yet are also selling like hot cakes.