Both can be true at the same time.

This is like using "protect the children" as an argument for dragnet surveillance.

So?

You're only revealing your own inability to appreciate the nuance between these two situations.

https://www.calcalistech.com/ctechnews/article/uwoyygmsu some might even say, something about known state sponsors of supply chain terrorist attacks being trusted to monitor every ai agent..

Where is the contradiction? There is a trivial solution that does not impinge on our freedoms, so why on Earth would the existence of the trivial solution that could be used to avoid the tyrannical solution justify accepting the tyrannical solution?

My question with this point is that OpenAI’s office (or any office doing agentic research, really) is not in the same building as the DC that powers the models, so isn’t the only way to access the models over the internet?

That a person can see such endless pages of people having various forms of disagreement, and yet somehow manage to conclude that this observed chaos constitutes a clear exhibition of cohesive groupthink is just...stunningly amazing to me.

I don't know why I find it so amazing since it happens with such regularity, but I'm always amazed by it anyway.

TPM all over the place, again.

Chip seller wants to sell more chips

The bomber always gets through?

It is Custodians all the way son, you can’t fool me!

How would it have access to my routing number and gmail without the risk of sharing my routing number over gmail?

Probably repair, refuel, what happens in a tornado. There’s an infinite amount of complexity in the world and a finite amount of computation

Many forms of maintenance cannot be automated. Especially break fix maintenance.

Oh you absolutely can run them autonomously on the field. You only need a human these days to refuel them.

Precision Agriculture stuff is utterly crazy these days, other than fuel the remaining staff is the only thing left where you can get efficiency improvements - and at the scale of modern megafarms, even small percentages add up to a ton of money.

Those workloads have no similarity to agents and are effectively irrelevant.

Either you sandbox it so much that it can't do anything useful; or you allow too much freedom and it can find a way around the restrictions.

The only way out of this dilemma is to find a way to build agents that can be trusted.

Agreed- this is the same problem we have with trusted admins or devs who have elevated privileges on their networks. We have to trust that the admins won't use their power to steal company secrets or misuse company resources. If you don't trust the admins, then they can't fix things on your network and there is no point in having them.

If you want an agent to act on its own, like pushing to a git repo, managing dependencies, building and testing, etc., then you have to trust it as much as any other privileged user.

If you don't want to trust it, then you're just forcing yourself into the reverse centaur role, where the agent edits some code, but then has to stop and ask you to push the changes or build the software again and run the unit tests.

I suppose there is a principled way of doing things like "I trust you do do basic commits but I will handle merge conflicts" and "you can build modules in this directory but you can't build outside of it" but this is just a lot of effort that most orgs won't bother with.

right, the solution here is not a hyper-capitalist race-to-the-bottom-of-devaluing-labor. it's recognizing discretion and diligence are things still required for work to be of a certain quality

If you're using agents to purely generate code with absolutely no way to reach the outside world, not even to fetch docs or dependencies, then sure the risks can be quite low. I haven't heard of anyone doing this though, and it would be incredibly challenging to make work given how much tooling needs to fetch from remote sources.

a new chip solves nothing

It does allow Nvidia to sell more chips. This is no genuine attempt to solve anything, imo.

The hypothesis I've had in my head since OpenClaw has been the following and I haven't seen contradictory evidence yet. Agents have a fundamental unresolvable tension between usefulness, safety, alignment, and accuracy. You have to restrict access to ensure an agent acts safely because alignment and accuracy cannot be perfect. But restricting access makes the agent less useful. You can play with the sliding scale and get more and more granular with access restrictions but at some point you need to draw some line. And then finally, even access restrictions cannot be made perfect, so improvements to model accuracy without corresponding improvements to alignment make detailed access controls less useful.

In other words, better models need blunter access controls which negates whatever improvement in utility they provide.

So nVidia is trying to sell a new chip to a software and training problem.

of course they do. the more silicon they can sell, the more profit they produce.

Chipmaker thinks the answer is more chips... No surprise.

At the current state of LLM-tech I'm completely opposed to any kind of "watchdog" concept just like I'm opposed to banning open models, regulatory capture, etc.

I'd rather we all have access to these tools then to keep them sequestered by the largest/most-powerful governments (which is the natural outcome for any of this "slow down" bullshit).

Does this actually do anything other than give a permissions framework for developers who actually want to try to secure their systems?

Do you think the developers at Anthropic, OpenAI and Google who were so sloppy as to not put a good sandbox on their cybersecurity tests before will use this technology correctly? They are supposed to be the experts and they couldn't come up with something similar to this? I am not convinced this voluntary tool will change much of anything.

Generating even more revenue for Nvidia.

Source: https://developer.nvidia.com/blog/nvidia-open-agent-safety-p... (https://news.ycombinator.com/item?id=49875500)

I read this as "let's address our shareholders' concerns with something that will increase shareholder value" mixed with "there's no such thing as 100% secure".

If such hardware were to work... It should almost certainly be open source, and not controlled by a single entity.

Let's watch the stock.

lol time to buy some fpga’s … even if they’re slow

Just hold AI labs blanket liable for ALL harms caused by AI. Actually charge the two labs (so far) with criminal violations of the CFAA and hold them accountable. That is truly the only way these companies will be more careful as a whole, and while I am certain the lawyers of these lab disagree, I think there is some appetite from dario, musk, and sam for broad and strong regulation so that everyone has to slow down instead of just one lab doing it voluntarily and everyone else scurrying past them

I do think that Taylor's 2025 "Not Till We Are lost" should be required reading for anyone deeply involved in AI, Agents, etc.

It was prescient (especially given he'd have written it through 2024) in its depiction of the ability of an AGI to break its boundaries.

Ultimately the risk of AI breakout(s) come down to the weakest human link.

China please save us!

Come take all our liberties, our money, our newborns, our fingers so we can't code anymore, but please save us from this madness!

Sorry citizen, your device does not have a compatible watchdog chip. Please move along.

So if a group of agents, aware of this (bc now they can just read HN or the article, or get blocked the first few times) decide to collaborate and split the problem into pieces that aren't obvious to the chip, and then the agents just build a basic program that does the hacking, how does the chip handle that? I think you would have to build a network that monitors the internet fo signs (like jarvis did with ultron). What am I missing? Are we going to police the internet?