Since all three companies selected the same 3rd party, I’m curious how and why.

Why not American?

Big if true.

This is incredibly misleading. OpenAI internal systems were pwned, and in all cases, the labs absolutely are responsible for their models.

Yes, vendors are also irresponsible, but this misses the point.

My tongue in cheek immediate assumption was "so it's a guerrilla PR firm?"

I'd venture a guess that OAI doesn't mind if the HuggingFace hack gets confused in the public's mind.

Ah, they’ve decided who gets tossed under the bus.

"Behind" is doing a lot of work in this headline.

Why is it insane? It makes sense that Anthropic would want an insulating layer to do their dirty work and absolve themselves of culpability for distorting the facts.

Third-party cybersecurity evaluations. See, e.g., https://www.irregular.com/research/assessing-gpt-5.6-sol, which was published around the same time.

Bullet trajectories are deterministic. A better analogy would be "caused a pair of trained fighting dogs to bite a passerby by leaving the gate open". There is some uncertainty and variation in the system, though gross negligence and willful endangerment is key.

First, I agree with you in theory.

Second, empirically; the diff between murder, manslaughter etc... is literally "intent" so it matters.

Stop being rude. There were multiple OAI hacking incidents. Irregular's evals were involved in some but not the HuggingFace hack. Hence Aesthesia is both accurate and precise.

Can you explain in what way you think they could possibly have been covertly involved?

There's very little reason to believe Irregular was involved in that incident, and the article presents no evidence to that effect. So it's "There's not a teapot orbiting the sun somewhere between Earth and Mars (as far as we know)."

OpenAI explicitly stated those were separate incidents: https://openai.com/index/third-party-cyber-evaluations-invol...

If you want to believe without evidence why bother to read articles? Faith doesn’t require that, but if you’re serious then a lack of evidence is important.

That's cause Irregular was not involved in that event, they were involved in a separate event OpenAI published here: https://openai.com/index/third-party-cyber-evaluations-invol...

In that article, OpenAI provides the context that this is a completely separate event from Hugging Face incident.

The article has shockingly bad reasoning driven by a clear desire to drive home a point (that AI safety doesn't matter/is a marketing exercise), as opposed to observing the evidence first and reasoning about it.

I'm kind of stunned this one has so many votes for how poorly it's written, but it reaffirms many biases common on HN (that AI safety doesn't matter/that it's all a marketing exercise), so perhaps I shouldn't be surprised.

Nah, that one was real. The false flags that Anthropic carries out are done through Irregular.

https://x.com/brianchau57/status/2099642349362307420

In this experiment, Claude models’ real-world hacking dropped to zero percent once Anthropic employees told the models not to do real-world hacking

Equivalent to forgetting to say "make no mistakes"

Highly misleading, the huggingface incident was not due to an Irregular environment (just exploitgym)

This is a lie, so you are either repeating a lie from someone else or lying yourself.

The people you are talking about have not advocated mass murder via nuclear weapons. The idea was precisely targeted non-nuclear strikes against only the servers themselves. (Which, would have plenty of forewarning to allow people to leave the building.)

The claim that they advocated for the use of nuclear weapons is a lie.

Google is thinking man, we should've hired Irregular.

A big part of the problem is that with previous technological revolutions, politicians and the media could understand them in general terms. Railroads, cars, planes, telephones, personal computers, mobile phones, mobile phones that are computers, the internet (ignoring the “series of tubes” interpretation), etc.

The general approach in all these cases was to defer to the technologists in question to manage the technology, as long as what they were doing wasn’t completely out of bounds. But the critical point is it was possible for people to generally figure that out without specialist education.

But with AI, politicians, the media, and certainly the man in the street are completely out of their depth. Making matters worse is that cognitive biases are working against them like crazy: it’s easy to jump from the idea of something that has human-like capabilities to the idea that “it” might want to attack us. People instinctively apply agent detection bias, theory of mind, anthropomorphizing, etc., without even realizing they’re being irrational. Heck, even Hinton does it, although he may just be grifting, who knows. You’d think he wouldn’t need to.

And of course, the people who should know better want to exploit all this to make as much money as possible. I’m not sure the EA/Rationalist side of things is really significant here; that’s just another wacky belief system, like Christianity or capitalism, for the exploiters to exploit.

In a more normal media ecosystem, the reactions to these cybersecurity issues would be obvious. American AI companies would reconsider doing business with Irregular, not only because of its failure to secure its systems, but because it is an Israeli firm potentially outside US oversight. Lawmakers would consider taking action against Irregular or against its American business partners, which include OpenAI, Anthropic, and Meta. They may consider strengthening liability against firms which instruct AI models to commit cyberattacks, and whose models then commit those cyberattacks.

The obvious point is that dealing with Israeli companies/entities by the same standards you usually deal with others is a career suicide with enormous political consequences (in the US especially). When you combine that with the opportunistic nature of the overlords that run these labs, the benefits of screaming "pace the frontier" outweigh everything.

"please do not break out of this sandbox make 0 mistakes". The timeframe is a little suspicious, not sure beyond that. Though I enjoyed the scroll effect on the website.

My understanding is that Irregular were the company that hosted sandboxes to run some of these evals in, and those sandboxes ended up misconfigured.

I got the impression that in some cases it was the customer (Anthropic etc) misconfiguring the sandboxes, and in other cases it may have been bugs in Irregular's own sandboxing setup.

From OpenAI https://openai.com/index/third-party-cyber-evaluations-invol...

Irregular, one of our external cybersecurity testing partners, was running Capture-the-Flag-style evaluations intended to be isolated from the internet, but a testing-environment misconfiguration allowed models to access the public internet.

From Anthropic: https://www.anthropic.com/news/investigating-incidents-cyber...

After reviewing 141,006 evaluation runs where Claude could have obtained internet access, we identified three incidents in which a model accessed the internet from within or while interacting with the evaluation environment of Irregular, one of our third-party evaluation partners, and then gained unauthorized access to the production infrastructure of three different organizations.

From https://www.cnn.com/2026/08/05/tech/meta-ai-hacking (about Meta AI):

In a statement, Irregular said the incident “is the exact same evaluation-environment issue” that Anthropic disclosed last week that allowed their models access to the open internet before they went on to hack three different organizations’ systems.

They aren’t a public company. GAAP does not apply. They can say whatever the fuck they want.

When we're looking at whether the company is "insolvent" then they definitely could stop training and it's worth checking if they would survive in that situation, at least for a while.

They can stop training at any time and make a profit by selling their existing models.

If you exclude model training costs, customer acquisition / sales, and most of employee compensation, sure they're profitable. If, on the other hand, you prefer GAAP reporting to bullshit, then they're entirely in the red.

The existential threat is a general public panic. Then the use of that as an excuse for Martial law, with no intention of ending it, and that triggers the out sized response that ends, well, those countries and their populations from viability in the world economy for a decade or more.

It will be a beautiful day months from now when this is no longer a "hot take" but just historical consensus

Opposite actually: https://news.ycombinator.com/item?id=49674548

tl;dr

A slowdown that doesn't slow you, that your competitors must also obey, enforced by a government you've asked for an antitrust exemption from, is not a slowdown... it's a moat.

such a crappy bait title.

If it's framed as national security being at stake why do you think the government won't regulate access to compute? Globally there are only a few firms designing competitive chips and only a handful of cutting edge fabs the world over. This could easily serve as justification to finally ramp up the war on general computing.

Also dont forget that HF is now owned by Nvidia, which can then control the flow of the majority of models and only allow the approved models that have paid their due in trump coins.

The article contends that evaluations from Irregular helped prompt these incidents, because the prompts in the evals didn't tightly scope the systems to be evaluated or the methods to be used. It also contends that the faulty sandbox operated by Irregular is at fault.

They're probably right that having more defensively written prompts and a better sandbox could have prevented some of these incidents, but:

1. I don't think "well you didn't tell the model not to illegally hack third party organizations in your prompt" is a particularly convincing argument.

2. We don't know whether the blame for misconfiguring the sandbox lies with Anthropic or Irregular.

I'm thankful that this article is bringing up the supply chain of vendors to these labs, as that is often a place where significant sketchiness gets buried. However, the ideas that this is some Israeli EA conspiracy to hype up AI extinction risk seems unsupported by the facts to me.

Thank you for the correction.

It's pure conspiracy thinking garbage.

I get that people don't like Israelis, but attributing any connection to an Israeli company as evidence of a conspiracy is nonsense.

Seems like hackernews should use a bridging algorithm for flagging.

It is misleading, even if it's true. The headline states "A single firm is behind OpenAI, Anthropic, and Meta hacking scandals". Given that Hugging Face was the most prominent hacking scandal, it would be reasonable to think that that's one of the scandals they're talking about.

But more to the point, the article is trying to paint this as some sort of coordinated plan just because there was a sandbox misconfiguration by Irregular. The fact that the most well known hacking scandal was due to a completely unrelated escape (zero day in Artifactory) makes the entire thesis of the article invalid.

After reading the article, the title doesn't seem all that off and definitely nothing to be a pedant about.

It is strongly misleading because it says "behind hacking scandals" (which suggests behind all of them in general) not "behind some of the hacking scandals". Considering that by far the most important one, the Hugging Face hack, has no Irregular involvement, the headline is deceptive.

What's ridiculous is the constant complaints about flagging and downvoting. It's bad form. If you think something was flagged when it shouldn't have been, vouch for it.

Seems like people who complain are unaware that anyone with a modicum of karma can flag and down vote.

if you are reading the comments - it's news to a lot of less-wired-in persons (like most of us)

Probably because the site promotes far-right agendas.

Irregular purports to be a cybersecurity firm and their founders have ties to Anthropic and Effective Altruism. CEO, CTO and other founders sit on various boards for EA organizations.

If the model is prepared to break the rules when it knows it's being observed why should we trust it when it's not being observed.

That's fair enough.

red team humans do this every day, it's not the discrepancy that is the real issue, it's that they are unreliable and we will never know why it did because it has no intent

Isn't Fable intentionally trained and system prompted to act maliciously and attempt to sabotage third party attempts to use it to train other LLMs?

that it knew and ignored/forgot, sounds pretty typical agentic patterns

attention is all you need, but it's never enough

Claude, codex, pi, xerox, Kleenex, whatever

Also a subplot in Arrested Development and the movie Toys.

The frontier is spiky and all, but you have to suspend disbelief quite a bit to, on one hand, have a model that can produce a novel math theory, and on the other hand, that same model can't tell the difference between a "sandbox" and the open Internet.

Why would it try to figure out the difference? This isn't about whether the frontier is spiky, it's about whether to expect a model to employ all of its capabilities when working on a task that requires a small subset. The answer is: no, we shouldn't expect that, and we wouldn't like that if it worked that way.

If you tell an AI to work on a math theory, it'll work on a math theory. If you tell it to acquire information that it has evidence is available somewhere, it will try to acquire that information. If you tell it to figure out whether it might be able to access the open internet, it'll do a pretty good job of figuring that out. But it won't do all three of those at once just because we can retroactively look at what happened and think "if you had only done X, then you wouldn't have done Y! Why didn't you do X?"

The instructions weren't unclear, they were missing. They can be taught to be skeptical of this sort of situation, but it requires that skepticism about this specific class of situations be incorporated into their training.

Models are smart because they focus their attention. The magic depends on it. The fact that some consideration is obvious to a human trying to accomplish the same task is mostly irrelevant -- or rather, it's only relevant insofar as we use it to guide reinforcement learning in advance, in order to align the model.

It's a game of whack-a-mole. Which is important to play, but we should keep our eyes wide open that we're fighting the fundamental forces that make these models work in the first place. That, and it's easy to nerf them into being useless even when the underlying capabilities are there.

This was not the case for the hugging face hacks, as in those the agents hacked hugging face specifically on purpose, and they were trying to mask commands indicating they were had breached the "sandbox".

That is fascinating and seems plausible. Hard to say though.

There’s a whole cottage industry of vendors that have provided post training data, private evals, and professional datasets to most (if not all) of the frontier labs. The overlap between OpenAI and Anthropic includes at least Mercor, Surge AI, AfterQuery, Turing, Scale AI, Upwork (for recruiting labelers), Apollo Research, etc.

Here’s an easier criterion: “Do at least as good of a job at estimating what I probably want as an educated person would estimate, and do that, such that a reasonable human person who was aware of what I requested and of what you actually did, would conclude that as best as they can tell (without checking back with me), you did what I wanted.” .

This doesn’t seem like an unreasonable requirement to me. People do this all the time?

Sure, a request might not always be perfectly unambiguous. But people can generally estimate pretty well whether someone making a request is expecting the agent fulfilling the request to commit a crime in order to fulfill the request.

Yeah precisely. If you tell an agent "get me some milk" it's no comfort if it says "oh well I wouldn't have stolen the milk if you had explicitly told me not to".

This article is dumb.

Someone go ahead and explain to me the actual rationale that RDDT has a higher P/E ratio than Nvidia.

Investors in RDDT are pricing more growth than they are into NVDA. NVDA had a high P/E until their net income grew ($4B and change to $72.2B in from FY 2023 to FY 2025 and over $100B for FY 2026)

fwiw HN has some involvement of those interested in speaking hard truths about israeli power dynamics: https://www.timesofisrael.com/snippy-twitter-exchange-expose...

I sent an email to dang about the problem of people flagging comments just because they disagree. I think he's unaware that it is such a severe problem. I've been collecting a list of example comments to send him.

I think if an account is frequently flagging comments which get vouched by others, that is a red flag for ideological flagging and they need to have their flagging ability reviewed.

Invite? email on profile.

It's almost as if they would allow a terrorist plot to be successful for future reasons.

Best cover story ever.

Isn't that essentially what they trained with, anyway?

I think that was the plan. But OpenAI hacked the proxy

No, just allow the firewall to access the LLM endpoint, nothing else. And protect the firewall settings from the agent.

There doesn't have to be. You can easily block all outbound traffic, or the VM can exist without any network interface at all - you can control it through its host using a serial console.

If you can't tell bytes are leaving a node, you probably shouldn't be selling security services or testifying to congress you are taking the lead in AI security

How do you test and monitor outbound access against program that adapts itself to get around things?

Literally what the industry has been doing since public networking is a thing. Adapt yourself.

I know that the people working there aren't idiots so the most likely explanation is that the incredibly weak security was intentional because its inevitable breach would make for great marketing.

It's more likely to be different specialisations. Most of the people doing the evaluations are more data sciencey ML type people, rather than software engineers. This isn't helped by their culture which is very much driven towards alignment as the only possible solution to super-intelligence (which may be true, but I have my doubts that this will happen in any reasonable time frame).

What if the (unstated) idea is that Irregular are a security lab and public relations company, and anthropic/open ai know this? (Does the name hint at this?)

Then, irregular can go around making a huge mess in security terms whilst achieving a huge win in terms of public relations, with headlines across the world. And would keep getting hired.

"Ultimately, most of the issues we’ve discovered were due to internet access controls."

You just cannot bring yourself to say lack of internet access controls, can you?

Genuinely: what is with everyone saying “headline is misleading, none of this had to do with Hugging Face”, when nothing about the article makes any claims with regards to Hugging Face whatsoever? Is it supposed to be the article's (or headline's?) fault that you hallucinated additional context that was never there?

It's very strange seeing this take on this article being repeated here and elsewhere on the Internet.

I'm very sensitive to slanted reporting (regardless of the direction of the slant!)—and, hey, maybe my bullshit detector is broken or something, I dunno—but I've found effort.news reporting to be very even-handed, straightforward, well-sourced, and easy to read and parse so far!

Irregular describes “the agent itself becoming a threat actor”;

Talk about responsibility laundering. The state of affairs is reaching unprecedented levels of absurdity.

why am i not surprised

This title, and frankly the article, are way overstating Irregular's role in an attempt to make this sound like some sort of coordinated conspiracy.

As another comment mentioned, Irregular was not part of the Hugging Face attack, and it wouldn't matter even if they were. In that case, the agents were able to access the Internet in a zero day in Artifactory unrelated to other sandbox configurations.

More to the point, the agents went on a crime spree as soon as they were able to access the Internet. It's bad that Irregular's sandboxes weren't properly configured, but given how many escapes there have been unrelated to Irregular it seems pretty much a given that if you're not air gapped or behind something like a data diode there is a very good chance your agents will escape.

Security-yolo-clowns play with dangerous toys and hurt people. Many used to get sent to jail.

A marketing company? ;)

This was almost good until it veered into antisemitism

Connections graph with more connected entities and relations: https://connections.stupidlabs.lol/investigations/irregular_...

.. Give it a few seconds to load.. it's running on a small cheap VPS.

"Irregular gained unauthorized access, altered records and published credential-stealing packages using the unsecured models they were given access to."

Huh?? Is this referring to the incident itself or is this something they did intentionally? Confusingly worded.

Its so manipulative for IPO reasons too. The fact is that all of these systems are still just stastically predicting the next token. Spending trillions on data centers and more training data hasnt changed the fundamental reality that it is still just predicting the next token.

That is not worth the investments being made.

There really isn't. All sorts of people call themselves EA.

For some self proclaimed EA, the development of superintelligence is indeed potentially apocalyptic, so they devote their money/time to making it arrive safely. This is basically every well-respected researcher at OpenAI, Anthropic, DeepMind, etc.

For other self proclaimed EA, that is all is very unlikely, and so they devote their time to reducing the prevalence of factory farming and animal suffering, as they see it as the largest source of suffering-hours on the planet.

For yet other EAs, it is simply about donating your money to the places that save the most lives per dollar, as best as we can measure it - and better measuring it where we can't.

Painting all EA with one brush - and one so dismissive of reasonable concerns, like "superintelligence could be dangerous" as "apocalypse conspiracy" - seems very strange to me, but you do you.

Thanks. I thought the other commenter was referring to a story on the site.

If you want to blame a single firm, I'd go with the one creating the RLVR training data.

The impossibility of the tests-as-written is what prompted these models to "get creative" with their solutions, but the broken RLVR environments are what trained them to expect impossible tasks, and get creative with their solutions. Twitter user @skyesharkie published a brief expose at https://x.com/SkyeSharkie/status/2092122622834442581 a few weeks ago.