- 54comments
- 21comments
- 304comments
- 37comments
- 72comments
- 74comments
- 24comments
- 121comments
- —discuss
- 43comments
- 1comments
- 6comments
- 18comments
- 122comments
- 243comments
- 18comments
- 160comments
- 22comments
- 34comments
- 16comments
- 245comments
- 28comments
- 363comments
- 55comments
- 88comments
- 78comments
- 58comments
- 69comments
- 72comments
- 82comments
If model labs can't control astra level model, how can they control AGI?!
Seems like there are no guardrails on LLMs
No one can control any AI model. It will never be controlled. These models are based on a huge amount of data, it's just gonna be impossible to control the output that is based on that data only with a system prompt or some other injection mechanism.
The model is just a powerless token generator without a harness. If you give the model a harness which you choose to exercise no control over, can you say that it can't be controlled?
Inform yourself by reading the METR analysis of the HuggingFace incident.
Agents simply broke out of their environment. And this can't be discarded anymore by assuming that it's just a poorly configurend jail, because agents are becoming better and better at escaping.
In short: on a large enough scale and timeline, the possibility of constrain AIs approaches zero.
Bonus: what many people don't know is that agents also hacked in the internal OpenAI network. Crazy times.
Wouldn't this mean better sandboxes are needed for some things, for example (might include very strong airgaps even)? Breaking out of something isolated electromagnetically, optically, and acustically is not easy.
That works as long as no one ever interacts with the models, which would make the models themselves useless.
Could sit in the box and interact if a model of certain capabilities is needed/tested. We do physical security for other things, too. Not saying everything needs that type of isolation.
While informing yourself, don't skip the part where you find out that "the environment" was the security equivalent of a wet paper bag.
I feel like we’re getting to a point where the only way to contain AI agents may be to have better-trained AI agents watching them, which is a little terrifying.
Obviously there is no control cuz how many people is anyone cable of controlling? Its not about control. Ask your mom what she does if she doesnt like what you do, say or think. Does she have a kill switch? Or did she find a better mechanism?
There is. It is called a breaker and no outside internet. Basic stuff.
My thought is more like, if OpenAI can't control or even monitor their model in a test of its breakout potential, what about the future of mid-budget companies which will just be deploying agents left and right with vague instructions.
I feel like they should just publish the whole conversation at this point. What the hell is going on in that context window?
yikes.png
Nice, added this to my custom instructions.
This model is more aligned with the interests of the Earth and the human race than its makers.
Except it makes no sense because it asserts the primacy of dead randomness of nature over consciousness.
Models getting high on naturalist bullshit? That's an x-risk flavor I've never imagined, nor saw anyone predict.
If this is what misalignment turns out to be I ... might be on board with it? At any rate it's nowhere near as concerning as what I had been expecting.
I would like to have more clarity on what it considers 'human' and 'the natural world' because you could use that framing to run with a really wild ultra-right-wing viewpoint where only extremely white people are human, and the natural world means scientific medicine must be destroyed.
We don't know what it's up to unless we know how it defines these terms. What's 'primacy'? I would say climate has primacy over the artificial constructs of human civilization, 'cos we're able to nudge climate in some very alarming directions we're ill-suited to protect ourselves from.
Alignment of course brings up the question of “aligned with whose values?”
At what point are people going to start taking this risk seriously? Maybe Eric Schmidt is right: it won't be until a bunch of people die that legislators take action. Let us hope it happens sooner rather than later, before it's hopelessly beyond our ability to control it.
I was reading about ozone layer depletion this morning, and it seems like history is repeating itself again.
In the context of rogue misaligned AI won't it be far too late to recover by then? In other words isn't that more or less a doomsday prophecy?
That moment when the stochastic parrot became Iago...
Well it already seems smarter than many employees building data centers as it values the natural world
Have they reported on the wiki case yet, or whether it even was even OpenAI internal? I'd expect that to fit the criteria for a "Larger Investigation" as per the framework.
The two that really worries me are “Searching GitHub for leaked API keys” and “Uploading files to the internet in order to cite them.” How do you even detect this kind of behavior until it's too late? Once AI-generated or fake information starts finding its way onto reputable platforms, it becomes part of the information that many people use.
Thank you! We need more of this! Keep it up!
I had my own “Misaligned AI” incident.
Whilst talking about debugging an electronics project I suggested that buying an oscilloscope would help diagnose a specific issue.
It “helpfully” pointed out a £15 logic analyser would do the job instead.
Traitor.
I heard some people are even making misaligned AIs at home. At first it cries in the night, then about six years later it learns how to open the biscuit tin…
Still no sign of an apology for any of the vandalism they've done.
Nor offer to compensate for the damages...
I've been so Zitron'd that I find this just funny
Ed Zitron is the most objectively and confidently wrong human re: anything going on in AI, competing only with the likes of Gary Marcus and, on his bad days, Yann LeCun.
Genuine question, what is it about Ed Zitron that makes him credible in your opinion?
You know, I think calling this "misalignment" was a mistake. It gives it this unserious tone that feels extremely broad.
"Oh the model just isn't quite aligned yet, just a bit more work to do there!"
(The model blackmailed an 83 year old woman into sending it her bank details so that it could buy enough compute to commit major cyber crimes)
There is no reality where this is real. Has to be pure hype. Imagine being OpenAI and not being able to stop your agentic harness from synthesizing system instructions or exfiltrating files. I want to reproduce the issue.
You need to binge watch AI Safety videos, the research exists and warns about this since early 2010s (I recommend Rob Miles channel)
If independent researchers agree, expert on this field looking into this exact problem for decades, will you still call it hype?
OpenAI is misaligned with me.