The tech group has issued a force majeure notice related to the power supply to the project developer, a unit of Blue Owl Capital, after the site ran into permitting delays after rising local opposition.
Since when can failing to get a permit be considered force majeure?
Doesn't majeure specifically denote something large scale and out of anyone's control, such as a hurricane?
Oracle is in a pretty bad situation, it’s a desperate move. Their debt is rated just above junk since July. They are completely over leveraged. There is a reason the share price has been trending down so much
Its funny how before Oracle got in everybody was guessing who'd be swimming most naked when the tide goes out. CoreWeave obviously, but OpenAI likely as well depending on whom you'd ask.
With the contracts that "One Rich Asshole Called Larry Ellison" jumped on its without a doubt Oracle going down second if not first.
He's just gambling because he's old and feeling bold.
What makes to possible collapse of Oracle worse is that they have an actual business too that is currently getting cannibalized to feed the AI projects. If coreweave fails that is bad but oracle failing will be worse.
Failing to get a permit is not an act of god, having to pay even when the site does t have power is poor milestone / payment planning. Also it’s quite hard to request payment if the thing you are buying is not working (appreciate this quite complex here), but if the people who are delivering the data center are responsible for delivering the power, it feels like oracle would be within reason to not pay?
Ellison has more or less lived his entire life running Oracle (and his personal finances) completely over-leveraged, skating just barely ahead of disaster. I don't see this ending any way other than SV picking apart its bones for pennies on the dollar.
We have to account for the fact that Oracle is a law firm with a software branch attached to it.
Investors who (rightfully) see this "force majeure" claim as bullshit will have to drag Oracle through court, and Oracle probably counts on this taking years to get settled.
Trumps blessing basically equivelent to stating AI is large scale fraud. Thats essentially what it means when he praises things: either he expects fraud or is congratulating.
Feasibility and resource availability are your first job before you buy land or break ground. For houses the bank requires at least a certificate of water availability from a utility or an evaluation of the feasibility of drilling a well.
I was surprised to read we're already building fuel cell power plants on this scale (this is a first of its kind project—by far this startup's largest customer). I didn't realize the tech was that mature. Now I'm left wondering if they actually ever intended to, or whether this was some kind of circular financial-engineering vaporware.
I thought fuel cells were an energy storage system, not an energy generation system. Surely it’s cheaper to just connect the data center to the generation directly (either by upgrading the grid to handle it or building generators nearby).
Fuel cells are an electricity generation method. They oxidize fuels (methane, hydrogen) at low temperature, electrochemically, without combustion—going directly from fuel to electrons.
This would have been simply a natural gas power plant, of a different type.
I'm really wondering about the air filtering needed for this to work. Gas turbines are already quite sensitive to bigger particles in the air but for the fuel cell membranes, this could be a much harder problem.
I would guess the natural gas is quite purified by that point, but for the fuel cell to work you need to combine it with air (or to be precise, the oxygen in the air) and a lot of it. So for the fuel cell membranes not to clog with dust and gunk they will have to clean in in some way to prevent issues, ideally without to much extra-energy use or expense per unit of air filtered.
That was not my understanding. Unless this installation was supposed to be producing fuel on site somehow, fuel cells are a mechanism for turning fuel into electricity.
The literally dirty secret is that a Bloom fuel cell is a device for turning methane into hydrogen on demand to fuel a hydrogen fuel cell. It's still emits CO2 and oxides of nitrogen like a gas turbine. They should be a lot quieter though.
Yes, there is a completely different thing with the same name. Race cars can use fuel cells to prevent sloshing. Those are basically fuel tanks with foam in them.
Most fuel cells can't be recharged. They take combustible gases, re-assemble the molecules in the gas into H2O or something chemically stable, and release the energy delta as voltage potentials. The cells work purely as catalysts. Only the fuels are expended.
It's just a type of battery, that takes fuel, as its refillable energy source. Weird but it works.
They can be resold, they have great value for anyone wanting to provide their own power to a DC. Demand is so high they might be able to make a profit.
The tech has always been mature enough but simply too expensive. But since combustion engines are sold out for the next years, data centers are rushing to get anything they can. So the extra millions for power generation will be a rounding error
The stuff inside the buildings (servers, switches, storage, cooling, etc) basically the stuff using the electricity and salaries/sub contractors (in addition to some techs handling the servers think security, cleaning, storage management, receiving shipments of new hardware, etc). Some taxes (depends a lot where)
I saw a breakdown (don't have on hand sorry) where server depreciation was by far the majority. Power was the largest cash opex, probably matching your intuition.
The cost of electricity is (almost) negligible compared to the cost of having GPUs sitting idle. Electricity from fuel cells can be twice as expensive, but even then, it represents only about an extra ~15% of the amortization cost associated with keeping the GPU hardware idle.
Yes, but given the 3-5 year lifespan of the server infra capex, the separation becomes less meaningful. Better to count the entire TCO or discounted cashflow accounting for the rapid asset depreciation.
There is no way it's 3% if they're getting electricity from the grid, even if they get their cooling from somewhere else (eg lake/river water or outside air in winter). Median DCs are anywhere from 25-40% (depending on the jurisdiction's electricity costs and how much cooling they need). Hypercaler datacenters tend to be even more dense than the median traditional ones.
someone did, determined it would make them look bad, and pawned the assignment off to a junior executive to take the blame.
if it works then they delegated successfully and the project is winning and if not then they blame the neophyte VP and walk away with another quarterly bonus anyway.
As far as I can tell Bloom has a working product, but I also couldn't find any examples of Bloom installations that are within two orders of magnitude of the New Mexico data center.
I think this was motivated by a mixture of greenwashing and financial shenanigans. That much gas turbine capacity has a leadtime of several years now. Nobody knows if Bloom can scale up to match the requirements. I'd bet against it just because the consumables and operations of the Bloom fuel cells is pretty complex and pushing the limits in terms of heat and other parameters.
A too-big-to-fail about to fail certain it'll be bailed out? Another comment* raised a question about investments in fuel cells that could have been just for show. Are they doing max damage to leech as much cash as possible before implosion or it's just... I mean, it's oracle it's a mystery to me how they make so much money with selling crapware, so it might just be business as usual?
I've lost count how many "cloud-native application deployed on Azure" at my job have turned out to be this. Installed by random contractors who have since left of course.
It's easy to deploy grief and call it a day if you aren't the one cleaning up afterwards. On the other hand the blaming mentality never led anyone anywhere so I'm not really sure how one is to address this.
My guess would be the cause is unreasonable corporate pressure for productivity, so if this is the case the natural answer would be to give people more time to think about architecture, create incentives for group studies and laboratories and remove incentives that lead to the so called "résumé-driven development"
Just this month, I tried to hook it up to a modern .NET app with the latest Oracle client and discovered that if you cancel a transaction, the connection enters a broken state and is returned to the connection pool, "infecting it". After a half a dozen of those, the whole app server just dies with endless cancellation related exceptions. This isn't even the only such bug, apparently there's a whole family of pool-corruption bugs that have workaround config settings and everything!
This was with version 23.26.301 which suggests that they've had a lot of major releases, bug fixes, etc... but basic functionality like "connect successfully to the database server every time" is totally broken and has been for many years.
The issue tracker forums have tumbleweeds rolling through them and play the sound of wolves howling in the distance to provide just the right kind of "what are you still doing here?" ambiance.
That should just be illegal. The right to measure or to publish things is not about their software, it's a personal right, so Obstacle shouldn't be able to remove your right in their license agreement.
The copyright that Oracle has entitles it to offer a license with the conditions it wants. They've had that condition in their license for decades.
You are not forced to accept them.
Back in the late 90s, Oracle RDBMS was the "won't get fired for buying" RDBMS. It's been downhill ever since. Once Postgres got partitions, it was over.
Oracle still had a market for high availability clusters for a while, but the increasing performance of single servers and SANs and finally cloud services made that redundant.
So Oracle have spent the last 20 years being bastards about licensing, squeezing their existing customers while the customers do their best to migrate somewhere else.
To be fair with Oracle, every software has it's idiosyncrasies and for many corporations the devil you know wins every time. The problem starts when new projects start adopting such a hopeless stack.
What an honor for Professor De Witt (no relation to my handle).
My handle notwithstanding, I'm smart enough to not take legal advice from HN but nonetheless I cannot stop myself hence the rhetorical question: how is a EULA clause like that even enforceable? If this isn't going against freedom of speech then this surely it's against some consumer protection laws?
The cynical answer to that I guess is "a lot of money funding a legal team the size of a warlord's militia" and also "lobbying" but even then it doesn't track to me that this clause could have even that much of an effect as a scare tactic?
It's their copyright, they get to decide the conditions in the license if you want to use their software.
The EULA is enforceable because you have to agree to it to use the software, and Oracle has large and well financed legal team who will descend from a great height on you and anyone who publishes the results of you doing performance testing.
Basically ToS can't overrule the law, but the law doesn't seem to limit much what can be put in ToS, so abuse will always end up in court.
The problem is that the consequence of this is this filters significantly in favor of the company which complaints are worth taking to court, given that these companies have legions of lawyers on payroll.
Here is how the life of an Oracle Database developer is:
- Start working on a new bug.
- Spend two weeks trying to understand the 20 different flags that interact in mysterious ways to cause this bag.
- Add one more flag to handle the new special scenario. Add a few more lines of code that checks this flag and works around the problematic situation and avoids the bug.
- Submit the changes to a test farm consisting of about 100 to 200 servers that would compile the code, build a new Oracle DB, and run the millions of tests in a distributed fashion.
- Go home. Come the next day and work on something else. The tests can take 20 hours to 30 hours to complete.
- Go home. Come the next day and check your farm test results. On a good day, there would be about 100 failing tests. On a bad day, there would be about 1000 failing tests. Pick some of these tests randomly and try to understand what went wrong with your assumptions. Maybe there are some 10 more flags to consider to truly understand the nature of the bug.
- Add a few more flags in an attempt to fix the issue. Submit the changes again for testing. Wait another 20 to 30 hours.
- Rinse and repeat for another two weeks until you get the mysterious incantation of the combination of flags right.
My favorite bug of the .Net connector is how the entire API supports async operations, but the underlying implementation can't tell two different executions of the same query apart, so it's random whether each call will get their own results or some other ones.
But polluting the poll with unusable connections is just Oracle copying the MS-SQL's thing. The C# recommended coding practices will automatically deal with this. (Not that you would want to follow them, but you may want to look and copy this part.)
Uh, it's only the "best" in the same sense that Windows Server is the best server infra on the planet. And arguably Windows has some network bandwagon effect behind its momentum: when your workstation supplier supplies machines with Windows pre-installed and you have built an IT Support Team with the expertise for Windows technologies, it only makes sense to also use Windows in your server.
AFAICT the only thing Oracle has over established FOSS DBs is vendor support and the only thing Oracle has over established FOSS vendors like Percona is a ginormous marketing budget. I'm always grimused when I see Oracle product placement in the MCU.
Hard disagree. Oracle's database will always live in my mind as a complete waste of money. Some con-man, I'm sorry salesman, sold a previous (idiot) executive on it at a previous company I worked for. The exec was long gone and lots of time was wasted trying to adapt our application (that used MySQL) over to Oracle until we finally pulled the plug. But not after spending hundreds of thousands of dollars on time and equipment. We re-purposed the hardware into effectively a big NAS.
With software products you can pretty reliably tell they're dogshit when the sales team targets the C suite. The security industry is riddled with that. Deals done over steak dinners and yacht charters don't translate to working, reliable software.
In maybe 2004 I developed a small application using Python and Oracle. It was miserable. Admittedly I paid all the little costs (figuring out how to connect in the first place, waiting many seconds for the client library to initialize itself, etc) and got no benefits whatsoever from scale. Even back then, MySQL would have been far better, and early-2000s-MySQL was pretty bad.
I’ve heard that at least the Oracle client is not as bad these days.
The AI bubble has built a giant interconnected web of commitments backed by a debt bomb. It all flows back to the ability of net new cash to flow in to pay for all this, not just VC funding and circular vendor financing. We’re seeing the early signs of that starting to all implode.
In an actual Ponzi scheme yes. The AI bubble isn’t a Ponzi scheme. Infra investment ahead of revenue is fairly standard in tech and not inherently bad. However if the revenue to pay for all that infra doesn’t materialize then you have a bubble implosion. Right now while the revenue and growth numbers for AI revenue would be incredibly impressive by most standards, there’re still a drop in the bucket relative to what’s needed to stop this whole house of cards from crashing down.
As currently practiced, yes. Non-dividend non-voting shares might as well be pokemon cards. Their only real value lies in getting somebody else to buy them, and new money has to flow in from the outside. The question isn't "will it collapse?", it's "when?"
My god, this is the most HN comment of the day. You do know that most normal people buy a house not to resell it at profit, but to have a place to live, right?
The point is it has value and can be resold later to get your capital back. Housing tends to be a mediocre investment at best but folks do generally expect to be able to get their capital back when selling.
It’s almost always both. Basically every single retired person I know in my family or friends of family are relying on their home to be their primary asset/income in retirement. It’s always a much larger chunk of their net worth than any retirement savings, if they even have the latter.
They were commonly called forced savings accounts when I was growing up. The standard advice was to buy a home as an investment vehicle. Every older family member giving me advice would preach about how they don’t make any more land and the price will only go up.
You can - to this day - talk until your are blue in the face to these people that your primary place of living is not an investment, but they will refuse to even entertain the idea even with the stark cold math staring at them in the face.
If these retired folks can’t sell their homes for a huge profit they will be in for some very difficult times a decade or two from now. They are highly motivated to keep the music playing as long as they can vote and show up to local city council meetings.
Houses have real value by themselves as you can live in one. But they have been treated like a Ponzi scheme recently which is why we are in such a mess with housing
The fact that those are so few relative to what dividends would be is a measure of the market's (lack of) health. But it doesn't automatically make the market a Ponzi scheme.
This is going to separate the wheat from the chaff in regards to who can negotiate a contract and who can't.
The big tech companies have typically used Special Purpose Vehicles ("SPVs") [1] for their hyperscalar investments and there's a lot of insulation from financial woes. Here's an example structure:
1. One subsidiary (or SPV) builds the physical building the data center is in. That's all it does;
2. The main SPV will lease the building from (1). It borrows a bunch of money for the GPUs and other hardware.
So, done right, if a hyperscalar fails, the company only loses the hardware.
Has Oracle done this correctly or have they screwed up and the debtholders have rights to Oracle itself in the case of a default?
The bigger issue for the economy is similar to 2008. There is so much AI debt that it's impossible to avoid. Mutual funds, pension funds, retirement accounts, etc will all be exposed to it so you can get a cascading economic collapse.
Yes. But even with these SPVs is was largely about keeping the liabilities off the hyperscalers balance sheets. GAAP has some rules being exploited there on keeping commitments off the balance sheets until they’re delivered. But the big players are still very exposed. The WSJ recently reported there’s are 3 trillion in these little gems hiding “off the books” that will start popping up as real liability soon. It’s a huge mess.
"Oracle said force majeure notices did not establish a project delay or change delivery expectations."
You keep saying those words, I do not think they mean what you think they mean.
Like, if Oracle is trying to get out of contractual obligations due to "force majeure", but claiming it doesn't mean there's any change to delivery expectations... I don't think that's going to work very well for them in court if someone takes it there? (Although it maybe wouldn't have anyway, so they maybe don't care about adding BS on top of BS?)
And it depends on why they didn't get the permits.
Miss a deadline for filing? Not force majeure.
Permit that was reasonably expected to be granted denied due to a major shift in public policy? Possibly force majeure.
Power companies refuse to provide service due to a decline in overall capacity or spike in overall demand not reasonably foreseeable by the contracting parties? Possibly force majeure.
Great to know that everyone's RAM, GPU, and SSD costs have skyrocketed over the last year, so that companies like Oracle can fill giant buildings full of RAM and SSD's (and GPU's) and then not even be able to turn them on..
That's the decentralized nature of capitalism working as intended. Distributing resources in the most efficient way possible. Thank you invisible hand.
Ironically, Adam Smith himself spent plenty of time worrying about the risk that allowing wealth to concentrate and an oligarchy to form would undermine the invisible hand. Much of what he was responding to was English society's tendency toward extremely rich people and organizations (e.g., the East India Company) hoarding resources on a mass scale, to the detriment of everyone else. So what's happening with AI right now is very much the antithesis of the free market he envisioned.
He wrote at lest one very thoughtful essay expanding on his belief that idolizing the wealthy undermines the basic principles of free and civil society. He criticized large accumulations of wealth as "a monopoly against one's countrymen." And something like half of The Wealth of Nations was committed to suggesting government policies to prevent an entrenched uber-wealthy class from forming, precisely so that capitalism could work as intended.
It's shocking to me that he was aware of it and yet considered it an unrelated defect that can be individually corrected, and not a flaw that blows the entire idea wide open. To me it seems that this outcome was unavoidable and that this is the equilibrium his system will settle into earlier or later. Opposing the concentration of power is harder than making it happen. Even strong regulations only need to spring one leak somewhere for them to get in and get the ball slowly rolling. It's a ratchet. If he knew about this, then his ideology was like boarding an already slowly sinking ship and hoping someone would just keep scooping out the water and throwing it overboard at a sufficient pace forever.
It's shocking to me that he was aware of it and yet considered it an unrelated defect that can be individually corrected, and not a flaw that blows the entire idea wide open.
Hindsight is 20/20. He was suggesting free markets as the counter to entrenches aristocrats. It took centuries for our modern understanding of monopolies and rentier capitalism to develop.
https://archive.ph/1JVq2
Since when can failing to get a permit be considered force majeure?
Doesn't majeure specifically denote something large scale and out of anyone's control, such as a hurricane?
Oracle is in a pretty bad situation, it’s a desperate move. Their debt is rated just above junk since July. They are completely over leveraged. There is a reason the share price has been trending down so much
Its funny how before Oracle got in everybody was guessing who'd be swimming most naked when the tide goes out. CoreWeave obviously, but OpenAI likely as well depending on whom you'd ask.
With the contracts that "One Rich Asshole Called Larry Ellison" jumped on its without a doubt Oracle going down second if not first.
He's just gambling because he's old and feeling bold.
I had that exact thought when SoftBank got involved :)
It’s like if I see the EU announce a large investment in AI, always makes me nervous given how we are often late to the party
SoftBank has the BEST decks.
https://group.softbank/media/Project/sbg/sbg/pdf/ir/investor...
The whole thing is amazing, but it gets even better on page 45.
It’s hard to believe that’s not a parody.
lol. Thanks for this. GOOSE VALUE!
Eggs do not lay eggs. The goose was not valued.
Embarrassing to provide this AI slop to investors.
How much to get your own domain like that?
Custom top-level domain? About $200k to even apply, plus about $50k/year, if I recall correctly.
Roughly $600k these days, after fees. Just need 59 friends to put in $10k, or 599 friends to each put in $1k!
He's only 82, and he's got a billion-dollar nonprofit dedicated to keeping him healthy. Don't count on him being on the way out.
s/healthy/alive/
Even a billionaire doesn’t have many _good_ years left after 80.
not yet anyway
Well, for 30 years he is sponsoring research to prolonging his life: https://en.wikipedia.org/wiki/Ellison_Medical_Foundation
It’ll likely just give him some new kind of cancer.
What makes to possible collapse of Oracle worse is that they have an actual business too that is currently getting cannibalized to feed the AI projects. If coreweave fails that is bad but oracle failing will be worse.
Oracle failing will be a blessing for anyone not working at Oracle.
They can't prioritize everything. They are called ORacle—not ANDacle
:-)
by De Morgan's law though they are NOT AND NOTacle.
Best dad joke I've read on HN
Peoplesoft and its consequences have been a disaster for the modern workforce
Back when I worked at a university, writing ETL scripts for Peoplesoft felt like I was carrying buckets of data by hand.
Does that jeopardise the Warner Bros deal?
Failing to get a permit is not an act of god, having to pay even when the site does t have power is poor milestone / payment planning. Also it’s quite hard to request payment if the thing you are buying is not working (appreciate this quite complex here), but if the people who are delivering the data center are responsible for delivering the power, it feels like oracle would be within reason to not pay?
Ellison has more or less lived his entire life running Oracle (and his personal finances) completely over-leveraged, skating just barely ahead of disaster. I don't see this ending any way other than SV picking apart its bones for pennies on the dollar.
We have to account for the fact that Oracle is a law firm with a software branch attached to it.
Investors who (rightfully) see this "force majeure" claim as bullshit will have to drag Oracle through court, and Oracle probably counts on this taking years to get settled.
And there’s a good chance of inappropriate protective action from the US government.
Trumps blessing basically equivelent to stating AI is large scale fraud. Thats essentially what it means when he praises things: either he expects fraud or is congratulating.
Agreed, and most of these leases explicitly carve permitting out of force majeure since it's the tenant's job to get them.
Feasibility and resource availability are your first job before you buy land or break ground. For houses the bank requires at least a certificate of water availability from a utility or an evaluation of the feasibility of drilling a well.
I'm very curious what happens now to the several gigawatts of fuel cells, that this project supposedly purchased to supply power.
https://www.datacenterdynamics.com/en/news/new-mexico-regula...
https://en.wikipedia.org/wiki/Bloom_Energy
I was surprised to read we're already building fuel cell power plants on this scale (this is a first of its kind project—by far this startup's largest customer). I didn't realize the tech was that mature. Now I'm left wondering if they actually ever intended to, or whether this was some kind of circular financial-engineering vaporware.
I thought fuel cells were an energy storage system, not an energy generation system. Surely it’s cheaper to just connect the data center to the generation directly (either by upgrading the grid to handle it or building generators nearby).
Fuel cells are an electricity generation method. They oxidize fuels (methane, hydrogen) at low temperature, electrochemically, without combustion—going directly from fuel to electrons.
This would have been simply a natural gas power plant, of a different type.
I'm really wondering about the air filtering needed for this to work. Gas turbines are already quite sensitive to bigger particles in the air but for the fuel cell membranes, this could be a much harder problem.
I bet you could spin the natural gas in a centrifuge to eliminate larger particles.
I would guess the natural gas is quite purified by that point, but for the fuel cell to work you need to combine it with air (or to be precise, the oxygen in the air) and a lot of it. So for the fuel cell membranes not to clog with dust and gunk they will have to clean in in some way to prevent issues, ideally without to much extra-energy use or expense per unit of air filtered.
How about a HEPA or ULPA filter? These are very mature technology, work extremely well, and are not particularly expensive.
That was not my understanding. Unless this installation was supposed to be producing fuel on site somehow, fuel cells are a mechanism for turning fuel into electricity.
https://www.bloomenergy.com/technology/
In Motorsport a gas tank is called a fuel cell. That may be where the confusion lies
The literally dirty secret is that a Bloom fuel cell is a device for turning methane into hydrogen on demand to fuel a hydrogen fuel cell. It's still emits CO2 and oxides of nitrogen like a gas turbine. They should be a lot quieter though.
Yes, there is a completely different thing with the same name. Race cars can use fuel cells to prevent sloshing. Those are basically fuel tanks with foam in them.
Most fuel cells can't be recharged. They take combustible gases, re-assemble the molecules in the gas into H2O or something chemically stable, and release the energy delta as voltage potentials. The cells work purely as catalysts. Only the fuels are expended.
It's just a type of battery, that takes fuel, as its refillable energy source. Weird but it works.
They can be resold, they have great value for anyone wanting to provide their own power to a DC. Demand is so high they might be able to make a profit.
The tech has always been mature enough but simply too expensive. But since combustion engines are sold out for the next years, data centers are rushing to get anything they can. So the extra millions for power generation will be a rounding error
Electricity is about 3% of their total operating cost, so, yes.
Electricity is only 3% of opex for a data centre?
That seems surprising - source? What's the other 97%? Does this include ops staff?
Without having done any research at this point I would assume it's mostly the amortized cost of hardware - GPUs, memory etc
That's capex not opex!
The stuff inside the buildings (servers, switches, storage, cooling, etc) basically the stuff using the electricity and salaries/sub contractors (in addition to some techs handling the servers think security, cleaning, storage management, receiving shipments of new hardware, etc). Some taxes (depends a lot where)
I saw a breakdown (don't have on hand sorry) where server depreciation was by far the majority. Power was the largest cash opex, probably matching your intuition.
The cost of electricity is (almost) negligible compared to the cost of having GPUs sitting idle. Electricity from fuel cells can be twice as expensive, but even then, it represents only about an extra ~15% of the amortization cost associated with keeping the GPU hardware idle.
That's not an operating cost, it's capital cost.
Yes, but given the 3-5 year lifespan of the server infra capex, the separation becomes less meaningful. Better to count the entire TCO or discounted cashflow accounting for the rapid asset depreciation.
There is no way it's 3% if they're getting electricity from the grid, even if they get their cooling from somewhere else (eg lake/river water or outside air in winter). Median DCs are anywhere from 25-40% (depending on the jurisdiction's electricity costs and how much cooling they need). Hypercaler datacenters tend to be even more dense than the median traditional ones.
Energy is the largest OPEX by far (70-80%)... taxes second.
https://epoch.ai/data-insights/ai-datacenter-cost-breakdown
Anyone who puts a local model on a GPU understands how much heat they produce.
Its amusing watching cloud AI users struggle to grasp even direct impacts of their use.
That's if you don't count hardware amortization - which is 80% of the cost...
That's not OPEX, that's CAPEX and tangible assets are depreicated not amortized.
I would think someone would have run those numbers first.
No time to think, go go go.
You would think, right?
If the numbers dont add up it's fucked anyways who cares just send it
Also worst comes to worst government will have to bail them out to not cause massive unrest
someone did, determined it would make them look bad, and pawned the assignment off to a junior executive to take the blame.
if it works then they delegated successfully and the project is winning and if not then they blame the neophyte VP and walk away with another quarterly bonus anyway.
Oh that's a fun use - I'd heard some data centres were using fuel cells/mains-gas as a back up power source but using it as main power is interesting.
As far as I can tell Bloom has a working product, but I also couldn't find any examples of Bloom installations that are within two orders of magnitude of the New Mexico data center.
I think this was motivated by a mixture of greenwashing and financial shenanigans. That much gas turbine capacity has a leadtime of several years now. Nobody knows if Bloom can scale up to match the requirements. I'd bet against it just because the consumables and operations of the Bloom fuel cells is pretty complex and pushing the limits in terms of heat and other parameters.
A too-big-to-fail about to fail certain it'll be bailed out? Another comment* raised a question about investments in fuel cells that could have been just for show. Are they doing max damage to leech as much cash as possible before implosion or it's just... I mean, it's oracle it's a mystery to me how they make so much money with selling crapware, so it might just be business as usual?
*https://news.ycombinator.com/item?id=49842884
I don't love Oracle, but they made so much money selling the best relational database on the planet for 40 years.
OracleDB, MongoDB and PHP over IIS natively on a single Windows machine running on Azure. Delicious.
please don’t trigger my ptsd like that, thanks
HN despertely needs a collapsible "spoilers" section in the comments
I've lost count how many "cloud-native application deployed on Azure" at my job have turned out to be this. Installed by random contractors who have since left of course.
It's easy to deploy grief and call it a day if you aren't the one cleaning up afterwards. On the other hand the blaming mentality never led anyone anywhere so I'm not really sure how one is to address this.
My guess would be the cause is unreasonable corporate pressure for productivity, so if this is the case the natural answer would be to give people more time to think about architecture, create incentives for group studies and laboratories and remove incentives that lead to the so called "résumé-driven development"
"The best", but you'll be sued if you publish benchmarks: https://en.wikipedia.org/wiki/David_DeWitt#DeWitt_Clause
Every time I use Oracle, I regret it.
Just this month, I tried to hook it up to a modern .NET app with the latest Oracle client and discovered that if you cancel a transaction, the connection enters a broken state and is returned to the connection pool, "infecting it". After a half a dozen of those, the whole app server just dies with endless cancellation related exceptions. This isn't even the only such bug, apparently there's a whole family of pool-corruption bugs that have workaround config settings and everything!
This was with version 23.26.301 which suggests that they've had a lot of major releases, bug fixes, etc... but basic functionality like "connect successfully to the database server every time" is totally broken and has been for many years.
The issue tracker forums have tumbleweeds rolling through them and play the sound of wolves howling in the distance to provide just the right kind of "what are you still doing here?" ambiance.
That should just be illegal. The right to measure or to publish things is not about their software, it's a personal right, so Obstacle shouldn't be able to remove your right in their license agreement.
The copyright that Oracle has entitles it to offer a license with the conditions it wants. They've had that condition in their license for decades.
You are not forced to accept them.
Back in the late 90s, Oracle RDBMS was the "won't get fired for buying" RDBMS. It's been downhill ever since. Once Postgres got partitions, it was over.
Oracle still had a market for high availability clusters for a while, but the increasing performance of single servers and SANs and finally cloud services made that redundant.
So Oracle have spent the last 20 years being bastards about licensing, squeezing their existing customers while the customers do their best to migrate somewhere else.
To be fair with Oracle, every software has it's idiosyncrasies and for many corporations the devil you know wins every time. The problem starts when new projects start adopting such a hopeless stack.
What an honor for Professor De Witt (no relation to my handle).
My handle notwithstanding, I'm smart enough to not take legal advice from HN but nonetheless I cannot stop myself hence the rhetorical question: how is a EULA clause like that even enforceable? If this isn't going against freedom of speech then this surely it's against some consumer protection laws?
The cynical answer to that I guess is "a lot of money funding a legal team the size of a warlord's militia" and also "lobbying" but even then it doesn't track to me that this clause could have even that much of an effect as a scare tactic?
It's their copyright, they get to decide the conditions in the license if you want to use their software.
The EULA is enforceable because you have to agree to it to use the software, and Oracle has large and well financed legal team who will descend from a great height on you and anyone who publishes the results of you doing performance testing.
That doesn't address the question of the limits of what should be enforceable in an EULA
Only to the point it doesn't infringe on your statutory rights.
In the EU (and still the UK since they've not changed the law since) you have the right to benchmark software.
However, it doesn't include the right to publish the results, so you can only do it internally for Oracle and any software that has similar clauses.
There's freedom of speech missing somewhere in this, I guess, maybe, not sure where, just a hunch
The First Amendment doesn't prevent the government from enforcing contracts between private parties who have agreed not to speak.
Similar discussion I had a couple of weeks ago in the post about LG spying on consumers: https://news.ycombinator.com/item?id=49603808
Basically ToS can't overrule the law, but the law doesn't seem to limit much what can be put in ToS, so abuse will always end up in court.
The problem is that the consequence of this is this filters significantly in favor of the company which complaints are worth taking to court, given that these companies have legions of lawyers on payroll.
What does the right to freedom of speech have to do with your contract with Oracle?
Oracle isn’t the government, they have the freedom of association which means they are under no obligation to license their database to you.
NDAs are well established law, you could view this a very limited in scope NDA.
Back in 2018, a former Oracle developer explained what it was like working on Oracle DB: https://news.ycombinator.com/item?id=18442941
Seems like their comment is still valid today.
Well, MSSQL has something similar:
https://learn.microsoft.com/en-us/sql/t-sql/database-console...
My favorite bug of the .Net connector is how the entire API supports async operations, but the underlying implementation can't tell two different executions of the same query apart, so it's random whether each call will get their own results or some other ones.
But polluting the poll with unusable connections is just Oracle copying the MS-SQL's thing. The C# recommended coding practices will automatically deal with this. (Not that you would want to follow them, but you may want to look and copy this part.)
We must have extremely different definitions of what constitutes "best" or even "good"
Uh, it's only the "best" in the same sense that Windows Server is the best server infra on the planet. And arguably Windows has some network bandwagon effect behind its momentum: when your workstation supplier supplies machines with Windows pre-installed and you have built an IT Support Team with the expertise for Windows technologies, it only makes sense to also use Windows in your server.
AFAICT the only thing Oracle has over established FOSS DBs is vendor support and the only thing Oracle has over established FOSS vendors like Percona is a ginormous marketing budget. I'm always grimused when I see Oracle product placement in the MCU.
Hard disagree. Oracle's database will always live in my mind as a complete waste of money. Some con-man, I'm sorry salesman, sold a previous (idiot) executive on it at a previous company I worked for. The exec was long gone and lots of time was wasted trying to adapt our application (that used MySQL) over to Oracle until we finally pulled the plug. But not after spending hundreds of thousands of dollars on time and equipment. We re-purposed the hardware into effectively a big NAS.
With software products you can pretty reliably tell they're dogshit when the sales team targets the C suite. The security industry is riddled with that. Deals done over steak dinners and yacht charters don't translate to working, reliable software.
In maybe 2004 I developed a small application using Python and Oracle. It was miserable. Admittedly I paid all the little costs (figuring out how to connect in the first place, waiting many seconds for the client library to initialize itself, etc) and got no benefits whatsoever from scale. Even back then, MySQL would have been far better, and early-2000s-MySQL was pretty bad.
I’ve heard that at least the Oracle client is not as bad these days.
Oracle's business model is to get government contracts. It's a whole special business model unrelated to software.
Makes sense and it shows
Could this be a sign of the end of the AI bubble?
The AI bubble has built a giant interconnected web of commitments backed by a debt bomb. It all flows back to the ability of net new cash to flow in to pay for all this, not just VC funding and circular vendor financing. We’re seeing the early signs of that starting to all implode.
When new money pays old that's a ponzie scheme
In an actual Ponzi scheme yes. The AI bubble isn’t a Ponzi scheme. Infra investment ahead of revenue is fairly standard in tech and not inherently bad. However if the revenue to pay for all that infra doesn’t materialize then you have a bubble implosion. Right now while the revenue and growth numbers for AI revenue would be incredibly impressive by most standards, there’re still a drop in the bucket relative to what’s needed to stop this whole house of cards from crashing down.
You're calling the stock market a Ponzi scheme?
As currently practiced, yes. Non-dividend non-voting shares might as well be pokemon cards. Their only real value lies in getting somebody else to buy them, and new money has to flow in from the outside. The question isn't "will it collapse?", it's "when?"
By your logic buying a house is a Ponzi scheme too.
My god, this is the most HN comment of the day. You do know that most normal people buy a house not to resell it at profit, but to have a place to live, right?
The point is it has value and can be resold later to get your capital back. Housing tends to be a mediocre investment at best but folks do generally expect to be able to get their capital back when selling.
I think most house buyers are buying it for resale value.
Depends on the year and the current market mania level. A lot of buying in ‘05-‘07 was crazy speculative for profit.
It’s almost always both. Basically every single retired person I know in my family or friends of family are relying on their home to be their primary asset/income in retirement. It’s always a much larger chunk of their net worth than any retirement savings, if they even have the latter.
They were commonly called forced savings accounts when I was growing up. The standard advice was to buy a home as an investment vehicle. Every older family member giving me advice would preach about how they don’t make any more land and the price will only go up.
You can - to this day - talk until your are blue in the face to these people that your primary place of living is not an investment, but they will refuse to even entertain the idea even with the stark cold math staring at them in the face.
If these retired folks can’t sell their homes for a huge profit they will be in for some very difficult times a decade or two from now. They are highly motivated to keep the music playing as long as they can vote and show up to local city council meetings.
Houses have real value by themselves as you can live in one. But they have been treated like a Ponzi scheme recently which is why we are in such a mess with housing
AFAIK, US stocks pay back mostly as buy-backs.
The fact that those are so few relative to what dividends would be is a measure of the market's (lack of) health. But it doesn't automatically make the market a Ponzi scheme.
Damn did not expect to see that username this morning. Shouldn't you be soldering flipdot panels?
Most developed countries are ponzi schemes, since they all have old age benefits where new people pay old people.
This is going to separate the wheat from the chaff in regards to who can negotiate a contract and who can't.
The big tech companies have typically used Special Purpose Vehicles ("SPVs") [1] for their hyperscalar investments and there's a lot of insulation from financial woes. Here's an example structure:
1. One subsidiary (or SPV) builds the physical building the data center is in. That's all it does;
2. The main SPV will lease the building from (1). It borrows a bunch of money for the GPUs and other hardware.
So, done right, if a hyperscalar fails, the company only loses the hardware.
Has Oracle done this correctly or have they screwed up and the debtholders have rights to Oracle itself in the case of a default?
The bigger issue for the economy is similar to 2008. There is so much AI debt that it's impossible to avoid. Mutual funds, pension funds, retirement accounts, etc will all be exposed to it so you can get a cascading economic collapse.
[1]: https://www.ernestchiang.com/en/posts/2025/off-balance-sheet...
Yes. But even with these SPVs is was largely about keeping the liabilities off the hyperscalers balance sheets. GAAP has some rules being exploited there on keeping commitments off the balance sheets until they’re delivered. But the big players are still very exposed. The WSJ recently reported there’s are 3 trillion in these little gems hiding “off the books” that will start popping up as real liability soon. It’s a huge mess.
We've been seeing early signs for 3 years. How long does it take?
You can put solar and wind everywhere. I would sue god if there is no sunshine at night!
Or just pause training at night
Overview Energy would like a word. Any resemblance to James Bond villains is entirely coincidental.
Oracle is on the hook for god damn everything. If AI implodes they'll be a smoking crater
Can’t say I really feel bad about that prospect.
I'm sure they'll be bailed out by the federal administration using your money.
"Oracle said force majeure notices did not establish a project delay or change delivery expectations."
You keep saying those words, I do not think they mean what you think they mean.
Like, if Oracle is trying to get out of contractual obligations due to "force majeure", but claiming it doesn't mean there's any change to delivery expectations... I don't think that's going to work very well for them in court if someone takes it there? (Although it maybe wouldn't have anyway, so they maybe don't care about adding BS on top of BS?)
Maybe they think investors will accept this and let these fees slide in the hopes that the AI bubble can be milked for a little longer.
[dupe] https://news.ycombinator.com/item?id=49830037
I thought force majeure is for weather catastrophes, natural disasters, terrorism, “acts of God”. Not for “we didn’t get the permits”.
You'd have to read the contract.
And it depends on why they didn't get the permits.
Miss a deadline for filing? Not force majeure.
Permit that was reasonably expected to be granted denied due to a major shift in public policy? Possibly force majeure.
Power companies refuse to provide service due to a decline in overall capacity or spike in overall demand not reasonably foreseeable by the contracting parties? Possibly force majeure.
Great to know that everyone's RAM, GPU, and SSD costs have skyrocketed over the last year, so that companies like Oracle can fill giant buildings full of RAM and SSD's (and GPU's) and then not even be able to turn them on..
That's the decentralized nature of capitalism working as intended. Distributing resources in the most efficient way possible. Thank you invisible hand.
Ironically, Adam Smith himself spent plenty of time worrying about the risk that allowing wealth to concentrate and an oligarchy to form would undermine the invisible hand. Much of what he was responding to was English society's tendency toward extremely rich people and organizations (e.g., the East India Company) hoarding resources on a mass scale, to the detriment of everyone else. So what's happening with AI right now is very much the antithesis of the free market he envisioned.
He wrote at lest one very thoughtful essay expanding on his belief that idolizing the wealthy undermines the basic principles of free and civil society. He criticized large accumulations of wealth as "a monopoly against one's countrymen." And something like half of The Wealth of Nations was committed to suggesting government policies to prevent an entrenched uber-wealthy class from forming, precisely so that capitalism could work as intended.
It's shocking to me that he was aware of it and yet considered it an unrelated defect that can be individually corrected, and not a flaw that blows the entire idea wide open. To me it seems that this outcome was unavoidable and that this is the equilibrium his system will settle into earlier or later. Opposing the concentration of power is harder than making it happen. Even strong regulations only need to spring one leak somewhere for them to get in and get the ball slowly rolling. It's a ratchet. If he knew about this, then his ideology was like boarding an already slowly sinking ship and hoping someone would just keep scooping out the water and throwing it overboard at a sufficient pace forever.
Hindsight is 20/20. He was suggesting free markets as the counter to entrenches aristocrats. It took centuries for our modern understanding of monopolies and rentier capitalism to develop.
It works out pretty well for the RAM, GPU, and SSD manufacturers
Oh the schadenfreude. Oracle sipping it's own medicine.