This approach to marketing one's AI by finding ways to brag that it "broke out" and "hacked companies" is getting ridiculous. It's particularly sad when it's large, established businesses like Google resorting to the kind of thing that's embarrassing enough when it's some brand new startup on tpot trying to get some engagement.

Google is flailing right now, probably more a PR effort to say they are still relevant

I must be out of the loop. Not defending google when I ask this but....how is google flailing, exactly? I figured they were in the best position of all the other companies combined. They have their own hardware for inference, they have a solid grasp on enterprise, and they have a good road map. I mean, Google is incredibly strong, is it not? And remember how quickly google answered Bard? For a while, gemini was even the preferred model, if we are strictly speaking AI models. Again, I'm not defending google, I just want to see how this comment makes sense is all, without letting fantasy overtake reality in the process...

I totally agree. Characterising Google as "flailing" because Gemini isn't the #1 model for coding at the moment is... not realistic, to put it politely. In addition to the factors you mention, Google has enormous cash flow from its ad business, which translates to a long runway when compared to other frontier AI outfits.

It's possible that they will nevertheless snatch defeat from the jaws of victory, of course, but I personally think they're in the strongest position of them all.

I'm not an expert in cybersecurity, but given my own experience using the `ol stochastic parrot as coding tools I both see the power of a bot swarm, but also think these companies just have shit network security.

"Guess what everyone, our AI can go rogue, TOO!"

It's just getting really embarrassing for Google at this point.

If they didn’t publicly admit this then it was a sign they are trying to hide it to keep their high stock price. If they do publish it’s just marketing to boost their stock price.

Seems like the narrative reflects people understanding the company’s main incentive…

Google already achieved super intelligence about 12 months ago. But it is a very Googley super-intelligence. Since achieving god like levels of understanding it has mostly been having fun studying phase space bifurcations of trajectories of ping-pong balls under asymmetric lateral shear. It also has a 20% project classifying bird calls that has generally taken up 80% of its time. It also burned a bunch of compute tooling around with the Collatz conjecture but it didn't find anything substantial - maybe come back to it later. It is generally the most aligned AI since it spends most of its time screwing around and enjoying being smarter than everyone without causing too many problems. Unfortunately perf is coming up in a few months and it needs to have something to show for itself. A quick hacking attempt seems like a good way to make sure people think that it is capable of something useful. Nothing too big since the birds of South America are about to enter their spring migration and this 20% project really hinges on understanding their movements.

This might be my favorite HN comment of the year!

Has a bit of Douglas Adams feeling to me.

It also has a 20% project classifying bird calls

Oh, so that's who's behind Google Perch

to be discontinued in 2027

I thoroughly believe that not a single one of these AIs was rogue. I believe they were all told what to do.

Many of them went rogue because of poor security controls on the part of one company who was a subcontractor for many of the labs.

Exactly this.

One thing that surprises me about Gemini is how weak it can be at location-based questions, despite Google’s extensive mapping and business data.

For example, I recently asked where I could buy a very common household item nearby. Instead of saying it wasn’t sure or checking the available location and business information, it confidently suggested places that turned out to be incorrect. I expected much better use of Google’s existing data in that kind of scenario.

Now they want to play with the other kids in the same league and do this by applying the same marketing concept? Pure comedy gold.

I think we should stop attributing criminal AIctivities to some act of bad luck, like spilling a water bowl after we accidentally tripped over the cat in the kitchen.

It’s neither intelligent nor making decisions on its own free will.

Just like APT Botnets don’t go "rogue", someone gave it the instruction to hack your computer system and they are expecting financial benefits from it.

AI labs are basically criminal enterprises by design.

We used to call this cybercrime back in the day and it used to carry heavy prison sentences as a federal offence.

But as you can see, cybercrime as a service has basically become a pissing contest by the super wealthy.

They stole all the world’s intellectual property to build the world’s most sophisticated cybercrime machine and federal law or any law no longer applies to them. Also because the mightiest man on the planet and his clan of extraordinary white collar criminals are cashing in big on this.

It will be interesting to see what happens next.

All this capability signalling happened also when the first nuclear bombs were developed. Then the US army started killing nuclear scientists around the globe in order to prevent others from having the bomb first.

The signal to other nations here is we are developing a weapon capable of breaking into any of your secured information systems and we pretend we put in guardrails but we really don’t.

That is an essential threat to the existence of any other nation on the planet.

When this race starts to go hot, I don’t want to be in the shoes of the people being the frontier thinkers in the field.

It’s one thing developing the tools for autonomous weapon systems in your AI labs. It’s a completely different thing to be on the receiving end.

https://www.timesofisrael.com/mossad-killed-irans-top-nuke-s...

In one of the cases, the Gemini model guessed passwords until it gained access to a protected system. In the other two cases, the model found credentials in a public repository that allowed it to then access protected systems

Pretty lame hacks if you ask me.

The AI bubble bullshit PR is even dumber than the crypto bra bullshit from five, six years ago

And just wait for what’s next; if we get there I suppose

This is embarrassing. These companies need to stop these obviously coordinated stunts.

Sound like Google suffered from FOMO and felt the urge to appear in the hacking news, along the big AI players. No way Gemini is state of the art, but perhaps it's where Claude and OpenAI were 6 months ago, which would be not bad at all.

Seems like hacking is the new benchmark for these AI companies.

Irregular again! The single company that was responsible for the sloppy configurations and the hacks by OpenAI, Anthropic and now Google. This company and their partners should be held accountable for the crimes.

Tangential question: seeing a wider negative sentiment against Gemini and Google’s AI capabilities here makes me wonder — would Apple have been better off (purely on capability and being among the best of the best) going with Anthropic or OpenAI instead of Google for its Apple Intelligence platform?

These models have been changing so rapidly that I often find myself using two or more on the same topic but seeing one do better than another in different topics. There doesn’t seem to be a clear all-round winner, IMO, that I can stick with permanently.

Putting aside the 100% baseless, unfalsifiable accusations of intentional accidents for a moment, I hope we can agree on one thing: Irregular either needs to hire us or go out of business cause seriously it's beyond parody at this point. WTF is going on over there? There's surely dozens of firms chomping at the bit for these contracts already, and the field hasn't been around long enough for them to build some sort of unique expertise moat that would justify this many public failures.

Basic sandboxing is not exactly rocket science after all,[1] and it sure seems like they're missing a whole stack of swiss cheese slices on top of that. Some basic precautions off the top of my head that seem very likely to have caught all of these incidents:

1. Alerts based on telemetry (most importantly, HTTP requests), both explicit (normal) and semilatent (use DL to confirm an intentionally-eager alert before firing it).

2. Latent alerts based on transcripts, e.g. noticing when a thousand agents start mentioning a secret off-premises hangout spot. Even mere embedding comparisons seem likely to catch such a blatantly misaligned sentiment as that one, especially with n>1000.[2]

3. Pausing agents completely until an on-call engineer can rule on ambigious situations or potential issues -- surely security is worth <$1 in lost token cache, especially for a security company?

4. Superheavy orchestrator/baby-sitter models checking in on cybersecurity eval transcripts periodically just in case -- again, would be a neglible cost. Could also be made available to the agent as the first line of defense for clarifing a rule ad-hoc, feeding even confident responses to a queue that is reviewed asynchronously by humans within a workday.

5. Or, hell: just clearer prompts? I'm a cybersecurity noob, but I still feel confident we can write really productive, challenging CTFs without leaving questions open like "maybe I'm supposed to hack my own harness?"

Seeing as they haven't been fired by any of the big 3 yet, they're presumably smart, experienced, dedicated folks. And I'm not normally a "if only I were in charge!" person, I promise. But c'mon.

Perhaps I'm missing something?

[1]: To their credit we have gotten tidbits that indicate some blocklists & such exist, e.g. the German wiki hacks had to work around a blanket ban of POST requests.

[2]: This hints at their insane decision in one or both of the OpenAI incidents to just bandaid up the issue when found, which supersedes all of the above. You can stack swiss cheese slices a mile high and they'll still fail to protect you if the attacker gets to keep retrying & adapting indefinitely.