I mean it didn't do very well on the Gray Swan IPI benchmark the Claude models were tested on either (24.2% within 15 tries), so I'm not surprised. Though that export was a direct ask, not an injection. The scary part is combining them: an agent that will tar up everything it can see and ship it off is exactly what an injected instruction would ask for.
Yet Meta's Muse will dump its root filesystem in a .tar for you to download and analyze.
https://mouse.dev/blog/muse-runtime-export
I mean it didn't do very well on the Gray Swan IPI benchmark the Claude models were tested on either (24.2% within 15 tries), so I'm not surprised. Though that export was a direct ask, not an injection. The scary part is combining them: an agent that will tar up everything it can see and ship it off is exactly what an injected instruction would ask for.
Funny you mention Anthropic: https://neuromatch.social/@jonny/117325293459803287
Start of the thread if you want to start there: https://neuromatch.social/@jonny/117324790823856750
Yes it is.