The big companies are insulated because they've locked in multi-year supply contracts with ram manufacturers - Microsoft, Google, Meta, Amazon (All on 3-5 yr contracts). OAI's deal with Samsung & SK Hynix is also colossal - locking up roughly 900,000 DRAM wafers per month, roughly 40% of world output (Stargate project).
Apple got caught out because it had a shorter contract that ended in the beginning of Q3, and they've tried to source their RAM from Chinese CXMT who declined because all capacity was already locked in contracts. They've gone with a lesser known company Kioxia.
Overall the consumer segment is completely neglected, companies don't care about end users in the current market conditions.
OAI's deal with Samsung & SK Hynix is also colossal - locking up roughly 900,000 DRAM wafers per month, roughly 40% of world output (Stargate project).
I wonder if they'll, at some point, have enough RAM? Or is this is the new normal? Will models keep scaling with the amount of ram chips openai and anthropic own?
Even with efficiency breakthroughs, it would just afford packing more agents per unit of memory. Scaling compute and data keeps paying off, leading to smarter models, and smarter models have more demand even at higher prices because they can accomplish more work at higher quality. Sovereign AI hasn’t even really taken off yet to anywhere near the level it could. That’s going to dramatically increase the number of massive-scale users of AI agents. So no, IMHO they will “never” have “enough.”
Hard to know, does each GB of ram give some marginal increase in profit or potential profit?
I’d guess no. Past a certain point the model has all the capabilities it can possibly usefully offer and honestly we may already be past that. The next gen model just doesn’t seem like as clear a step up as it once was.
and they've tried to source their RAM from Chinese CXMT who declined because all capacity was already locked in contracts.
The US Government also wouldn't let Apple use Chinese RAM anyways, even for product that was just going to be used in China. China does have capacity issues though, and focusing on local brands first is probably the right call. Hopefully they can ramp even if they can't get the fancy lithography machines from the Netherlands.
Overall the consumer segment is completely neglected, companies don't care about end users in the current market conditions.
Mirroring trends throughout the entire economy (not just RAM and other inputs for AI).
Everyone is chasing the top 10% or higher of the K-shaped economy for all goods and services, servicing everyone else isn't seen as being worth the investment.
This will just keep getting worse and worse everywhere for everything as long as we allow income inequality to keep exploding, which seems to be the plan.
I wonder how this plays out with automotive? Seems like it could get messy, especially if automotive rated parts are decided to not be worth the premium.
"companies don't care about end users in the current market conditions."
Because the end users keep rushing to use the megacorp's latest AI models, weaving them into their work and life. So the megacorps keep locking in contracts to build more compute.
If you use LLMs, you're responsible for this, there's no way to pass the buck. "I only use it as a companion for learning about history" - it's still your fault. "I only use it to help guide my solitions, not for vibecoding" - it's still your fault.
And how do you think it will work? Do you really believe that once competition to serve top 10% get really fierce, no company will try to capture the remaining 90%?
China's pretty good at bending the curve. I expect that by the end of 2027 we'll see a lot more available for consumers. As others have pointed out, this might not be available in US markets.
"CXMT currently has two 12-inch DRAM fabrication plants — or fabs - in Hefei and one in Beijing, with a combined capacity of about 300,000 wafers per month.
With the new Shanghai facility and other new capacity, CXMT will double its DRAM wafer output to approximately 600,000 wafers per month, all three sources added."
Overall the consumer segment is completely neglected, companies don't care about end users in the current market conditions.
It's almost like these companies WANT a dystopia with a centralized winner-takes-all power structure. Anything in the name of profits, who gives a fuck about humanity and distribution of rights or freedoms.
On the bright side, this will further accelerate the retro computing scene as people throw their hands up at the thought of building something shiny and new
I think it's time we start talking about putting limits on how much compute AI companies can purchase or own, relative to the rest of the world. It's not fair that they can use trillions of dollars of investor billionaires money to consume all of the resources that everybody needs. Where is fair distribution? What about all of the industries they're destroying in the process by hoarding it all for themselves?
Otherwise, there will be no end to this. There are no hard limits on the speed of a parallel bruteforce. It's an infinite complexity problem class. The more parallel bruteforce power you have, the more likely you are to be able to solve a problem. So there is no world where demand ends. So if something isn't done about this, we'll have million dollar GPUs and RAM sticks because they've priced everyone out of the market and are the only ones able to afford them. Say goodbye to owning your own hardware at that point.
I called this almost a year ago that the way things are going that home users and pc enthusiasts will be priced out of computing. And the best thing for the rest of the world is China reaching on parity on node side and crash the market. What would be the prices today if Chinese companies were not forced to build their own chips.
we're driving consumer prices up and delaying the climate transition for Rube Goldberg slop machines. In case any future historiographer decides to go with the title "the age of intelligence, our stupidest period yet" I'd like some royalties
Pretty soon we'll see a renaissance in the tech that we gave up years ago or can still be improved
- memory compression algorithms
- alternative LLM architectures that don't rely on memory or GPUs
- compatibility hardware (like DDR3 to DDR4 boards)
- distributed computing improvements, both at local GPU and networking levels (SLI for AI)
- GPU hacks to add more memory or support older architectures
I'm personally looking forward to the new LLM architectures that don't require as much compute, e.g. DLLMs, which can be good enough for CPU usage but lack the accuracy of frontier models currently.
When this happens the bottom will fall out of the GPU and memory markets, putting a glut of cheap hardware out there.
The big companies are insulated because they've locked in multi-year supply contracts with ram manufacturers - Microsoft, Google, Meta, Amazon (All on 3-5 yr contracts). OAI's deal with Samsung & SK Hynix is also colossal - locking up roughly 900,000 DRAM wafers per month, roughly 40% of world output (Stargate project).
Apple got caught out because it had a shorter contract that ended in the beginning of Q3, and they've tried to source their RAM from Chinese CXMT who declined because all capacity was already locked in contracts. They've gone with a lesser known company Kioxia.
Overall the consumer segment is completely neglected, companies don't care about end users in the current market conditions.
I wonder if they'll, at some point, have enough RAM? Or is this is the new normal? Will models keep scaling with the amount of ram chips openai and anthropic own?
Even with efficiency breakthroughs, it would just afford packing more agents per unit of memory. Scaling compute and data keeps paying off, leading to smarter models, and smarter models have more demand even at higher prices because they can accomplish more work at higher quality. Sovereign AI hasn’t even really taken off yet to anywhere near the level it could. That’s going to dramatically increase the number of massive-scale users of AI agents. So no, IMHO they will “never” have “enough.”
Hard to know, does each GB of ram give some marginal increase in profit or potential profit?
I’d guess no. Past a certain point the model has all the capabilities it can possibly usefully offer and honestly we may already be past that. The next gen model just doesn’t seem like as clear a step up as it once was.
The US Government also wouldn't let Apple use Chinese RAM anyways, even for product that was just going to be used in China. China does have capacity issues though, and focusing on local brands first is probably the right call. Hopefully they can ramp even if they can't get the fancy lithography machines from the Netherlands.
Formerly known as Toshiba's memory division. They spun off the business as Kioxia in 2017.
Kioxia is Toshiba, The most known company from the list, and it doesnt make any ram
Mirroring trends throughout the entire economy (not just RAM and other inputs for AI).
Everyone is chasing the top 10% or higher of the K-shaped economy for all goods and services, servicing everyone else isn't seen as being worth the investment.
This will just keep getting worse and worse everywhere for everything as long as we allow income inequality to keep exploding, which seems to be the plan.
I wonder how this plays out with automotive? Seems like it could get messy, especially if automotive rated parts are decided to not be worth the premium.
"companies don't care about end users in the current market conditions."
Because the end users keep rushing to use the megacorp's latest AI models, weaving them into their work and life. So the megacorps keep locking in contracts to build more compute.
If you use LLMs, you're responsible for this, there's no way to pass the buck. "I only use it as a companion for learning about history" - it's still your fault. "I only use it to help guide my solitions, not for vibecoding" - it's still your fault.
And how do you think it will work? Do you really believe that once competition to serve top 10% get really fierce, no company will try to capture the remaining 90%?
China's pretty good at bending the curve. I expect that by the end of 2027 we'll see a lot more available for consumers. As others have pointed out, this might not be available in US markets.
"CXMT currently has two 12-inch DRAM fabrication plants — or fabs - in Hefei and one in Beijing, with a combined capacity of about 300,000 wafers per month.
With the new Shanghai facility and other new capacity, CXMT will double its DRAM wafer output to approximately 600,000 wafers per month, all three sources added."
https://www.reuters.com/world/china/chinas-cxmt-wins-3-billi...
This is probably my ignorance but why are we assuming that CXMT capacity will also not be gobbled up for datacenters?
It's almost like these companies WANT a dystopia with a centralized winner-takes-all power structure. Anything in the name of profits, who gives a fuck about humanity and distribution of rights or freedoms.
Its a good thing the data centers in the Gulf states didnt get bombed.
On the bright side, this will further accelerate the retro computing scene as people throw their hands up at the thought of building something shiny and new
When will supply catchup with demand?
I think it's time we start talking about putting limits on how much compute AI companies can purchase or own, relative to the rest of the world. It's not fair that they can use trillions of dollars of investor billionaires money to consume all of the resources that everybody needs. Where is fair distribution? What about all of the industries they're destroying in the process by hoarding it all for themselves?
Otherwise, there will be no end to this. There are no hard limits on the speed of a parallel bruteforce. It's an infinite complexity problem class. The more parallel bruteforce power you have, the more likely you are to be able to solve a problem. So there is no world where demand ends. So if something isn't done about this, we'll have million dollar GPUs and RAM sticks because they've priced everyone out of the market and are the only ones able to afford them. Say goodbye to owning your own hardware at that point.
I called this almost a year ago that the way things are going that home users and pc enthusiasts will be priced out of computing. And the best thing for the rest of the world is China reaching on parity on node side and crash the market. What would be the prices today if Chinese companies were not forced to build their own chips.
we're driving consumer prices up and delaying the climate transition for Rube Goldberg slop machines. In case any future historiographer decides to go with the title "the age of intelligence, our stupidest period yet" I'd like some royalties
The good thing is that we no longer consider RAM is free and forced to develop more efficient program (hopefully)
What about Samsung Hynix cartel? Were they punished already?
Pretty soon we'll see a renaissance in the tech that we gave up years ago or can still be improved
- memory compression algorithms
- alternative LLM architectures that don't rely on memory or GPUs
- compatibility hardware (like DDR3 to DDR4 boards)
- distributed computing improvements, both at local GPU and networking levels (SLI for AI)
- GPU hacks to add more memory or support older architectures
I'm personally looking forward to the new LLM architectures that don't require as much compute, e.g. DLLMs, which can be good enough for CPU usage but lack the accuracy of frontier models currently. When this happens the bottom will fall out of the GPU and memory markets, putting a glut of cheap hardware out there.