Yes, exactly, imagine if there was a sudden and unprecedented in scale plague of cattle escaping containment and causing car wrecks all over Wyoming and Montana, and multiple incidents of AR-15 armed sheriff deputies having to dispatch them on site.

1. How are you distinguishing boasting from responsible disclosure and calling for safety standards?

2. If during their early days, cigarette companies or oil companies had published information & financial filings about the risks of smoking/climate change and calling for safety standards, would you similarly call that “boasting”?

3. If Chinese AI labs similarly published info whenever their models did unexpected or malicious actions, would you similarly call that “boasting”?

Yeah but your neighbors car is somewhere in the tens of thousands of dollars, maybe more, but a model hacking and being malicious could be worth anywhere from thousands to millions and God forbid... BILLIONS in damages. Models were NOT doing these sorts of things 1 or 2 years ago as far as anyone knows.

This is not some theoretical, there are lots of existing laws about who is liable for damages caused by livestock.

Some areas are open range. If you don't want cattle on your land its your job to put fences up to keep them out. Other areas are restricted to livestock and it's on the rancher to keep them out of where they shouldn't be.

“A computer can never be held accountable, therefore a computer must never make a management decision.”

– IBM Training Manual, 1979

Also, when people operate equipment know to cause harm when accidents happen, such as cars, they're required to undergo training and testing before being granted a license to operate them, and to have insurance to ensure other people can be made whole in case of damage or injury. And there is an organization and profession that monitors people's use of that equipment, handing out fines or license suspensions based on actual or even just potentially dangerous operation.

Sadly, software "engineers" are not held accountable for their profession like everybody else with a real engineer title. And AI companies are all building Ford Pintos as fast as they can, and competing with each other about who's product makes the biggest explosion.

This is a whole lot more obvious once you stop anthropomorphizing LLMs.

Timescale is a relative measure.

If corporations are people then language models should be dogs.

Then we jail politicians if it messes up. If that doesn't work, we start jailing voters

They all hired the same startup security firm Irregular to try their internal unrestricted models, which is weird especially for Google to do. At OpenAI, they didn't stop the test even after it was apparent the bots had broken out. That's all I got.

What’s the question? Why model output isn’t always what you expect it to be?

Consensus is just populating the model with rationale for different new output.

I didn't downvote you, but your last paragraph is unnecessarily aggressive.

I've worked with agents, and I agree with you that often there isn't another way to express the interactions.

However, I also think the terms ML uses in general are a mimicry that misleads people who aren't informed. Ask anyone outside SWE what they think “training” means, and they'll usually picture something being taught.

I don’t think anybody can change that now, but it’s useful to point it out.

You're discussing how to speak _to_ a model, but everyone else here is discussing how to speak _about_ a model.

I didn't downvote, but wow you're being aggressive, you have a lot to learn if you read the comments here with an open mind.

It really does feel like a purposeful thing on the part of the big labs.

For the most part it feels like most people are waking up to it though.

Regarding:

However, all the attention goes to the human-like conclusions in agent thinking traces, which creates a misperception of sentient AI for people who don’t know how the magic black box works.<

I know there’s been some questions regarding if thinking traces are even relevant to the outcome most of the time.

I think you’re jumping the gun on my response a little. I’m just telling you what the purpose could be.

Now do I think that’s the reason? It certainly isn’t a new thing for companies to try to do that. Shift blame that is.

Regarding civil liability, I’m not making that argument here. But it makes sense from a public perception viewpoint why they would want the agents to appear at fault instead of their own actions.

Right, I don't dispute that their PR language minimizes their culpability in public opinion, but I don't see how it impacts their liability in court.

Was not groundwork laid when people were talking for decades about legal entities using such language? Look at the posts here. Microsoft always acts like that, Meta like this, Apple did that, Nvidia never does this. In a tone that ascribes agency and accountability to a company name. Even though it is always individuals that are responsible, not "company".

Well, here we are.

I wish I had the slightest clue how to dial down the insanity and approach this whole situation with reason and logic, but the people in charge aren't up for any of that foolishness, and far too high a percentage of everyone else chooses to either 100% believe every word of the crazies in charge as Gospel Truth, or at the other end of the extreme, wanting to outright shut down the technology entirely due to "doomer" fears, and none of them want to listen to a single word from anyone who actually understands how this stuff really even works. It's not freakin' magic FFS! It's math and software. Some of us do know how it works and try desperately to help others understand, but nobody wants to understand. They just want to believe the "machine god" is gonna save them (or kill them). It's just a tool. Whether it does good or bad depends entirely on what we humans choose to do with it and how well we use it (just like any tool).

Have you ever seen someone claiming ai labs should have no liability in the wild? I haven't

I'd describe David Sacks's position as, approximately, no limits on AI. At some hypothetical future state, sure, maybe, but right now, nothing. That's tantamount to consequence-free action.

I think a lot of the developers in these big labs barely understand how it works

Yeah, but the more common ongoing cases will likely be end users whose agents perform the hacking.

Indeed, and similarly, if a craftsman injures themselves or a co-worker with a perfectly good (non-faulty) tool, then it's entirely on the craftsman. Same goes for "AI". Misuse the tool, suffer the consequences.

1. If the tool maker implemented reasonable attempts at safeguards I guess that’s a point towards the toolmaker not being liable.

2. If a toolmaker did not build in safeguards, I guess that would mean it’s more likely they’d be liable.

Civilly, it's fairly clear. Criminally, it's clear too, just not in the direction you want it to be. Criminal liability for hacking requires human intent; not recklessness or negligence or even knowledge without giving a shit, but provable intent.

this is 100% an already solved problem.

the idiocy here is the stupid psudo "AGI" marketing around the services.

its really simple. Whoever run the service to do the task requested is responsible. If OpenAI sent an agent out to train their AI then the directors are to be held responsible, if a user of the service used the service and it inadvertently "hacked" someone then both are held responsible.

throwing AI into the mix changes nothing about how the law is applied. its a tool, like a car or a gun. The user of the tool is responsible for how its used, the manufacturer is also responsible for the safety of it.

It’s a question, because a lot of money is riding on people developing a mental model of a llm being something else than a computer program.

"Back in 2022, a Google employee already thought their AI model was sentient."

Sigh, this meme again

I don't see this as being much different from a crane operator or airline pilot.

One of my clients has enforced a policy where a live human user principal must be supplied as a header with any requests outbound from the AI system. The effective policy is that you are completely (100%) responsible for what your agent does on your behalf. The AI system is designed to request confirmation for any potentially destructive actions.

Doesn't seem like hard dilemmas... Would you use the word "use" for hopping on a bus ? Well same for hopping on an autonomous car. Unless you touch the wheel you didn't use it. Same for the second example, doesn't seem ambiguous as all, the hardest part I guess is proving it that there was no malicious intent, but with logs and all that, doesn't seem that hard.

Okay Isaac Asimov: how do you define "use"? If I pay for an autonomous car, and I sit inside it while it drives around with A.I., am I using that A.I?

If you're just sitting in a car and using it for its intended purpose you wouldn't be accountable for the AI doing something that causes harm.

If some 3rd party, without your knowledge tells AI to act on something you said in video and a reasonable person would expect harm to result from that, the 3rd party would be accountable but you wouldn't be.

If A.I. agents are in my notebook computer, or they are in a cloud server where I have an account, or they are somehow acting while I have them at the command line, but they spontaneously act whether or not I command them, and they work in the background and they work without prompting, but they can also be commanded by direct user prompts... are we using those agents?

AI agents never "spontaneously act". They have no desires or goals beyond what they are told to do. If you aren't aware of what they were doing, and a reasonable person wouldn't be expected to know what they were doing, you wouldn't be accountable if what they did resulted in harm.

Its been well established that blame is distributed in an inverse proportion to the various parties wealth/power/status metrics. The higher these metrics, the lower the accountability.

Nobody cares. Seriously, beyond navel gazing on social media, nobody cares.

By the time it’s an actual problem and not just these guys trying to use it for viral marketing, you’re going to have many thousands of people doing it maliciously with intent to worry about. You’re going to be flooded with Russian hackers with no recourse.

"Who's responsible for training an assassin and asking it to go out into the world ?"

The fact this is being discussed as a legitimate question is the real story.

"We trained this beast of processing power, we asked it for a task, and it did something wrong... Who's to blame ?"

Trained on stolen books and material, every word we've all spoken, most lines of code we've ever written with not even an acknowledgment.

Must be the data scientists in their rooms calculating the response rate of every token to blame ? Our version of AI is not sentient. Stop making it seem so. But we need to ask where to look for the culprit ?

In the end, the only job left was liability.

A major problem with LLM's is that they don't reason in a way humans are used to thinking of reason. If we tried to give them something like Asimov's laws of robotics, they likely wouldn't be able to apply them reliably. This is a challenge for AI companies working on the bleeding edge, and it's fairly obvious those companies should be held accountable for mistakes, whether caused by carelessness or not. It's no different than an oil spill. They may or may not be subject to charges based on what happened but, regardless, they are responsible for cleanup costs.

What's less obvious is who should be held accountable when a customer of one of these corporations uses their product and it unexpectedly does bad things. e.g. A fellow asks his AI assistant to book him into a high-demand class at the local gym, so the LLM probes the gym's website for vulnerabilities, books him into a date that is farther into the future than the system is supposed to permit, and then drops other people from earlier classes until he's bumped into the one he wanted. If the gym decides to press charges, who should they be applied to?

This sort of case is more difficult to answer. The company that provided the AI certainly bears some responsibility. Perhaps most of it. Possibly even all of it if they represented their AI as reliably law abiding. If a user knowingly uses an AI that is not guaranteed to abide by the law, is that user partially liable for what the AI does too?

IANAL. I'd love to hear perspectives on this question.

We had the same debate when self driving cars started to be a thing, and we decided that the companies making the self-driving tech are responsible..

So if an AI agent is asked to build a giant base for someone in MineCraft, and decided to build a swarm of additional agents, and one of those agents says "Time to destroy all humans" and autonomously hacks into the pentagon and fires the nukes - the company that developed the model is responsible. That being said - if the nukes deploy successfully, I have two questions:

1. If no one finds out, is anyone responsible?

2. Was any of this actually real?

Obviously, the labs (or any other operator of a model) should be accountable for malicious or destructive actions taken by agents.

And they are. I don't think there's any controversy about the civil liability exposure frontier labs have if their agents cause damages, and it is remarkably easy to rack up damages by causing computer intrusions even if those intrusions don't cause obvious direct damages; for instance, many organizations are required to engage forensics firms at nosebleed-high costs to assess the impact of breakins in order to retain insurance coverage.

The "controversy", if you want to call it that, is over criminal liability. People feel that frontier labs should be at least as responsible criminally as human hackers are when they're caught (to be clear: an extraordinarily rare outcome).

The problem is: they're not criminally liable, not so long as the frontier labs operate without specific intent to cause breakins. Mens rea thresholds are their own whole area of criminal law, and there are stark differences between "recklessness" and "intent". All of the meaningful criminal CFAA predicates require actual intent: someone, a human being, has to deliberately set out to create the outcome where a specific intrusion happens. They have to want it to happen and act accordingly. In the most severe cases, they also have to do so with intent to defraud.

We could change the law to make it easier to prosecute breakins without provable intent, but I don't think that would make HN people happier.

At the core, how are these agents any different from what Aaron Swartz was driven to suicide for?

In both cases, someone ran some software that maybe called other software that ended up doing an action which was possibly illegal.

Downloading journal articles is worthy of punishment but compromising multiple websites is worthy of… heady press coverage?

Should be both the entity providing the platform where the inference is running + the entity giving the prompts/instructions.

I’m pretty sure that this is a solved problem. For “classical” machine learning, it worked like this in my neck of the woods:

The model’s operator is directly liable for any undue harm caused in the course of the model’s operation. This includes models acquired from third-party vendors. The operator is responsible for ascertaining the model’s fitness for purpose prior to deployment, and for ongoing monitoring of its operation.

If the model came from a vendor, and the operator conducted due diligence but it turns out that the vendor materially misrepresented the model’s capabilities in a way that contributed to the harm, then the vendor can also be held liable.

If that happens then it’s up to a court to apportion the liability.

IANAL but I see no reason why these principles shouldn’t apply to GenAI.

Obviously the most proximate human being in the decision-making chain that led to the AI agent being deployed. The nearest person with the power to say no who said yes.

The CEO, CTO, and the board of directors should be held criminally liable.

Unless the money and decision makers are held liable, there will be no impact on the direction of the company.

The person who ordered it or allowed it?

Seems pretty simple to me.