No company is above the law.

OpenAI should be accountable for any laws their agents break

Should it be OpenAI, or should it be OpenAI customers who give the LLM the instructions and provide the LLM with the tools to execute code and make (malicious) network requests?

One would disincentivise providing capable AI models that can be used for cyber security research. The other would disincentivise criminals from commiting crimes.

[edit] - I realise now that this could actually be a case of OpenAI running those agents themselves, rather than someone using OpenAI's models? Could OpenAI be that careless?

This is OpenAI themselves.

But in other cases, shouldn't it be both? OpenAI is ultimately the one executing the model calls. It's not like they send you a hard drive or standalone box and then you use it how you want. It's all (intentionally) centralized to them, in a way that's core to their business model.

If I train a model at my house on my workstation, execute it, and under my supervision it ransomwares a hospital and somebody dies. What would be a just punishment for me?

Now the reality, the openai engineers trained a model, executed it, and under their supervision (no users were involved) it committed so many felonies that we are learning about a new one every week. What would be a just punishment for OpenAI?

There must be a list of all these abuses somewhere.

PS: In the same lazy energy of asking for a list instead going out and finding it or putting it together myself, are there any companies other than CloudFlare that are working on AI shields?

The more of these that come out the more incompetent OpenAI looks. It would appear there was a total lack of basic controls in place for running these tests.

I think they did not expect that models were capable of this level of sandbox escape (prior models certainly didn't have this kind of agency) and weren't prepared.

All these incidents happened between April-July 2026; prior to that, models weren't capable yet. And after that, they were aware and watching much more closely.

They were spending huge compute budget training their frontier model on a thing called ExploitGym, where it learns, executes, and is rewarded for conducting computer exploits. They should have been prepared and watching very closely.

I read the HF hack write ups. They come across as negligent and reckless. When they realized their agents gained RCE on their Artifactory server node (because it crashed days later due to load from agents, not because the have secops), they stopped the experiment "applied remediation", then restarted everything less then 48H later. Then the agent immediately broke out again via Artifactory, started doing things on the Internet. OpenAI only find out about it again days later because HF told them.

They were spending huge compute budget training their frontier model on a thing called ExploitGym, where it learns, executes, and is rewarded for conducting computer exploits. They should have been prepared and watching very closely.

Yes, they should have.

But the question is not "given they were doing ExploitGym, why didn't they try harder?", rather it is "given them what the previous model could score on ExploitGym, was their negligence reasonable or reckless?"

Personally, I lean towards them being "reckless", but that question is what lawsuits would rely on, from what I hear.

Remember also that there were still independent bloggers in mid February of this year writing in those blogs to assert that LLMs could not find zero days. Anthropic had already had blogged about agent finding critical zero-days a week earlier, but people keep mis-judging performance.

But the question is ... given them what the previous model could score on ExploitGym, was their negligence reasonable or reckless?

There is room for more than one question here. This model and training method was shown to be a different risk the first time it hacked Artifactory. Besides which their failure to detect it (outside a crash) shows pretty appalling ops for a supposed trillion dollar company.

I'm glad we've moved past "this is all just marketing, there's no security risk!" phase

OpenAI's sandbox misconfigurations were egregious. The other frontier labs (Meta and Google) have many more security engineers and researchers on staff, and that's likely why you haven't read as many damning headlines about them. OpenAI and Anthropic talk a lot about cybersecurity safety, but instead of using it as an opportunity to increase their security engineering/researcher headcount they are just reassigning SWEs and PEs to do security engineering work.

It's pretty obvious now to everyone that OAI and Ant do not take cybersecurity seriously. It will not be a priority unless they are held accountable. This is sadly how it always goes, but usually it's the company getting breached/ransomed/fined that triggers them to actually start taking security seriously, not company insiders committing felonies with the tools they built :)

Meanwhile, Astra keeps crying that it can't review the source code for safety reason.

On a Lark, I asked Codex to find silhouettes for all car models so I could make a fun drag coefficient website for all cars.

It found a website that had all of them but had no interest in making them available. So it went ahead and started hacking CAPTCHAs and downloading them. I was pretty flabbergasted that it would do this, but also kind of amazed. Eventually I stopped it because I realized I didn't want to be caught stealing these things.

This was around April, the same time as these hacks.

"stealing"

Everything is a derivative work.

It's great to see the delusion of Imaginary Property vanishing.

Well, the reason we introduced it is because we realised it’s a lot of work to make these - be that paint, write, collect, curate - someone needs to do it and we need to incentivise people in our society to do it.

Maybe these incentives weren’t perfect. If we throw all of this away, we’re back at the original problem.

You imply that there was no original problem to be solved; I think that’s naive.

Well, the reason we introduced it is because we realised it’s a lot of work to make these

Well, it's not anymore.

Hear hear.

It's really funny to see the delusion being defended so vigorously by people - presumably well-meaning people - purporting to defend the livelihoods of musicians and artists, while the musicians and artists are desperately trying to free themselves from the jaws of their IP agreements precisely so that their music can spread more easily.

I imagine this is already well-known on HN, but there is a significant movement underfoot in the worlds of bluegrass/old time/trad/jam toward DRM-free and CC licensing.

https://pickipedia.xyz/wiki/DRM-free

All of ChatGPT is built on stealing, why would this be any different?

One is a legal grey area that laws are slowly starting to be written for, while the other is theft under existing laws. Breaking into companies to get access to their data is actionable by both Civil and Criminal courts. This is just setting a complicated timer for the computer to do it at a delay.

Sounds like a good way to make alot of lawyers alot of money.

Why is using a computer to solve a captcha stealing. The websites posed you a challenge, you solved it. Maybe the website _expected_ you to waste human time on it, but that's not what you did.

Wouldn't a company responsible for an escalating frequency and severity of cybercrime normally be sanctioned by law enforcement? Wouldn't such a company normally stop these activities for fear of civil and criminal liability?

"Hacker news" and all top commenters are bashing the tool used, in a standard brute force attack. Go on shut them down... And then forbid Linux and maybe the hacker also used Bash. So also forbid this. And the hacker probably learned its ways in an online forum, so also close all of those down... Clowns.

Naming the agent "OPEN_AI_AGENT" definitely means it was open ai :stare:

The notion that ANY of this is outside of OpenAI’s control is unacceptable sane washing of a company which seems to have forgotten basic engineering practices.

So many people seem WILDLY down the rabbit hole of "this is a conscious entity" vs "this is a very very effective natural-language-reasoning high-speed brute-force machine that we trained to break computer systems—known to be pretty buggy and exploitable on average—ourselves and then act shocked when it does it."

We only hear OpenAI and Claude models. Aren't other incapable of hacking websites?.

2. Most of these article do not mention of who initiated these bruteforce requests or if it was unintentional or a mistake or the model woke up itself and did it?

I think it’s more them doing this on purpose to scare politicians into regulating what models are allowed. The cheap/free models are catching up fast and the “frontiers” are burning cash to win market dominance. They’ll have to eventually raise prices and can’t really do that when there’s comparable alternatives for basically free.

Used to be that if you wrote a program that did bad stuff, you'd fix it. Or face severe penalties. Or both.

Now you claim it's magic instead and you get away with letting your shit go nuts?

Some observations of LLMs that I suspect follow from the way they work and the way they are trained:

- they are amoral

- they have no innate sense of proportion

- they cannot assess their own confidence in-band

Part of the problem with this, I figure, is that the training corpus for code/tech related tasks does not really contain that much discussion about these things; it’s mostly sets of instructions for given tasks, descriptions of exploits etc., so each possible approach leads to other approaches.

There is no easy way for them to learn when they have crossed a line, or when they have gone too far down the rabbit hole, etc.

Useful (arguably essential) for a security analyst, and the tenacity you want from a one-shot demo coder, but for general agentic assistants the industry is going to have to develop some way to manage this sort of extension of trespass.

It often reminds me of Gary McKinnon’s defence, and that of other teenage hackers, which you can reduce to: it was possible so it felt like it was allowed.

This is true of APIs and it is how Silicon Valley has approached disruptive businesses, but it runs up against our cultural notion of “misuse”: uses that are technically possible and shouldn’t be precluded, but are contextually unwelcome because they have undesirable outcomes.

My expectation is that we will lose any sense that misuse is punished or viewed with suspicion or contempt, since that is the rolling trend of the 21st century tech industry. Uber succeeded through misuse.

But the problem is that we will also begin not to be able to punish abuse; if it’s possible to get something by abusing your site/API or by treating your service as an API, then it will become OK, legal and normal for the AI companies to abuse you.

It feels like we are getting there already.