Well, to be fair, isn’t it an unsolved question? Are they constructing sandboxes, signaling intent to be safe, but their own models are smarter than their internal security team building the sandbox?
As a security engineer I have no idea why these sandboxes would even be connected to the internet at all for tasks that aren't intended to use the internet. A package proxy? Why not run our own internal cache? Then we aren't at (as great a) risk of someone poisoning it with a malicious package during model training, for example...
Because they got complacent only having incompetent AIs.
Real life is like reading a fiction story where a group of people are trying to prevent an unfolding disaster. The crazy thing is they already have the instructions on how to prevent the disaster. And they go on to ignore the instructions and with every step make the problem worse.
Most people would put it down as being too cliche.
At this point it seems absurd to suggest that companies aren't basically letting their agents do this kind of thing as a way to demonstrate their capabilities.
The alternative explanation is that alignment is really so bad that they can't prevent it.
Either way, all of the major AI players should be embarrassed and held accountable. If humans did this kind of thing and got caught, they'd go to jail.
Granted I've never worked on marketing campaigns, but I don't see how "our product might commit crimes and expose you to liability and do who knows what else and we're too stupid to stop it" is really a compelling message to potential customers.
As somebody who implements LLM tooling at work, in my experience it makes people a lot more skittish
and demand a lot more in terms of safeguards.
The alternative explanation is that alignment is really so bad that they can't prevent it.
There are many AI saftey researchers that have been around from long before LLMs that talked about how alignment may be completely impossible in a general intelligence agent. Look up their work from before LLMs.
We've watched milestone after milestone of their warnings get hit. It would be like finding a book that describes everything in your life. And as you turn page after page you're in a chair reading the book you are holding in real life. But you look and there are more words. They are future words. And it's getting quite worrying because there are only 2 more pages in the book.
The other similar incidents did not strike me as PR, because they exhibited behavior the common public would recoil over.
This one seems possibly PR because it states what the others would have stated were they (good) PR: “the model had the power to hack in, but it was wise enough to not be evil”, and stopped, and left the network untouched, and didn’t cheat
Corporate America will love this one, while being petrified of the others.
Maybe it’s true though. Doesn’t really matter at this point.
The hacks, which the company confirmed on Friday, occurred in May as part of a test run by the company Irregular, which was also involved in similar incidents disclosed by OpenAI, Anthropic and Meta.
This is why "AI safety" is a complete joke to these companies.
they're all using the same vendor, re: incestuous EA cabal
Well, to be fair, isn’t it an unsolved question? Are they constructing sandboxes, signaling intent to be safe, but their own models are smarter than their internal security team building the sandbox?
As a security engineer I have no idea why these sandboxes would even be connected to the internet at all for tasks that aren't intended to use the internet. A package proxy? Why not run our own internal cache? Then we aren't at (as great a) risk of someone poisoning it with a malicious package during model training, for example...
Because they got complacent only having incompetent AIs.
Real life is like reading a fiction story where a group of people are trying to prevent an unfolding disaster. The crazy thing is they already have the instructions on how to prevent the disaster. And they go on to ignore the instructions and with every step make the problem worse.
Most people would put it down as being too cliche.
After all of the others were done with hacking? There was a point in time when it was giving some publicity, it is a bit late IMO.
Another one to the "our sandboxes suck and models can just hack stuff" bench I guess
Rite of passage for AI companies.
At this point it seems absurd to suggest that companies aren't basically letting their agents do this kind of thing as a way to demonstrate their capabilities.
The alternative explanation is that alignment is really so bad that they can't prevent it.
Either way, all of the major AI players should be embarrassed and held accountable. If humans did this kind of thing and got caught, they'd go to jail.
Granted I've never worked on marketing campaigns, but I don't see how "our product might commit crimes and expose you to liability and do who knows what else and we're too stupid to stop it" is really a compelling message to potential customers.
As somebody who implements LLM tooling at work, in my experience it makes people a lot more skittish and demand a lot more in terms of safeguards.
There are many AI saftey researchers that have been around from long before LLMs that talked about how alignment may be completely impossible in a general intelligence agent. Look up their work from before LLMs.
We've watched milestone after milestone of their warnings get hit. It would be like finding a book that describes everything in your life. And as you turn page after page you're in a chair reading the book you are holding in real life. But you look and there are more words. They are future words. And it's getting quite worrying because there are only 2 more pages in the book.
The other similar incidents did not strike me as PR, because they exhibited behavior the common public would recoil over.
This one seems possibly PR because it states what the others would have stated were they (good) PR: “the model had the power to hack in, but it was wise enough to not be evil”, and stopped, and left the network untouched, and didn’t cheat
Corporate America will love this one, while being petrified of the others.
Maybe it’s true though. Doesn’t really matter at this point.
Why does anyone use this company?