I don't get it. Why is breaking the law so hard to enforce when it is a company (i.e a person or group of people consenting to) running a computer program? If I take a gun and spray bullets around me I don't get to write it off as the gun being dangerous.
Many web people got their start freelancing. You may know some. It is not uncommon to incorporate when doing that. The idea is if you fuck up their site somehow (or more likely - they claim you did) they can't try and go after your personal bank account or your car or your house. Only what is held by your company. You also get other benefits, like tax breaks for being such an industrious member of society.
The same incentives are being set up on both sides. Because generally it is good for society when people start companies. A company as a liability shield is only a bad thing if you think personal assets should be at risk because someone is unhappy about what you did at work. If someone wants to sue you over something that happened as part of business deal, why should they be able to threaten the house your family lives in?
Seems sensible to me. While there may be problems as the scale of the company (and remuneration of its officers) increases, the underlying principle isn't exactly hard to understand. If you shot up a public area and then tried to claim that was a business activity, you would obviously not have an argument. It isn't plausible on its face. So I'm really not sure what point you're making.
If it's just complaining that it is hard to prove things in court when massive companies are involved, then tough. It is also relatively hard to convict people who shoot off guns in public even if a whole bunch of people saw them. In simpler times a mob would have simply formed and executed them. We don't get to do that anymore, because we don't want to live under mob rule.
> If you shot up a public area and then tried to claim that was a business activity, you would obviously not have an argument. It isn't plausible on its face. So I'm really not sure what point you're making.
I'm running a terrorist training simulation and unwitting participants entered the area! Whoops! You haven't addressed the point of a company (composed of people) creating something that exceeds its bounds. Is a company liable if they are working on nuclear power and delete the city they are operating out of? Why would a software company be treated differently when it causes harm to a third party?
> I'm running a terrorist training simulation and unwitting participants entered the area! Whoops!
There is already precedent on more or less this exact scenario. Except you usually don't call people under your own jurisdiction "terrorists". That sort of term is reserved for people running those businesses in certain other places. By the way, it is an interesting feature of the American legal system that this sort of incident could spawn many proceedings, where a civil suit against the company in question could be one among many.
> Is a company liable if they are working on nuclear power and delete the city they are operating out of?
In theory, obviously. You still have to prove it in court.
> Why would a software company be treated differently when it causes harm to a third party?
They wouldn't. Again, you still have to prove it in court.
What point are you making? This hasn't been brought to court yet and I am lamenting the fact that charges have not been brought.
> There is already precedent on more or less this exact scenario. Except you usually don't call people under your own jurisdiction "terrorists". That sort of term is reserved for people running those businesses in certain other places. By the way, it is an interesting feature of the American legal system that this sort of incident could spawn many proceedings, where a civil suit against the company in question could be one among many.
I don't have a jurisdiction as I am not the judiciary nor law enforcement. I'm really not sure what you're driving at.
> I don't have a jurisdiction as I am not the judiciary nor law enforcement. I'm really not sure what you're driving at.
If you were shot or a member of your family got "deleted" and you had sufficient control of the estate, then you would have standing.
Obviously, if you aren't involved you shouldn't be able to sue and be compensated because something happened to someone else that doesn't relate to you at all.
>If you were shot or a member of your family got "deleted" and you had sufficient control of the estate, then you would have standing.
> Obviously, if you aren't involved you shouldn't be able to sue and be compensated because something happened to someone else that doesn't relate to you at all.
Crimes don't require standing wtf are you on about? Prosecutors bring charges not individuals.
> Prosecutors bring charges even if the estate doesn't pursue.
Simply wrong. In criminal cases, the government may not elect to bring charges for all sorts of reasons. The civil courts will still be available to you. That will be true whether there are charges or not. Whether that results in a conviction or not.
The government can also initiate a civil suit. Criminal charges against businesses are rare. I'm not sure any would apply here. Perhaps you'd like to volunteer a statute?
> It is not uncommon to incorporate when doing that. The idea is if you fuck up their site somehow (or more likely - they claim you did) they can't try and go after your personal bank account or your car or your house. Only what is held by your company.
This is a common misconception. A corporation doesn’t protect you from personal liability. If you intentionally or negligently damage someone’s property they can sue you personally. In many cases they can sue both, and it’s easier to collect from an established business, but that doesn’t do you any good as a freelancer.
Nuisance lawsuits aren’t much of an issue because insurance covers the legal defense and indemnifies you
You're talking about limited liability companies, which is a legal structure relevant for civil claims.
Someone committing crimes "on behalf" of a company is not protected by the corporate veil and absolutely will get prosecuted individually (well, for small companies -- big companies get to play with different rules, which was the point GP was making).
Also the corporate veil is not absolute, it can get pierced in a few cases, and not everyone incorporates when they do contract work (in which case you need indemnity / private liability insurance).
The best is that the article frames it as "Autonomous hacking is here. Governments are not ready". As if someone didn't authorizing turning the damn thing on.
> TO LOSE CONTROL of one artificial intelligence may be regarded as misfortune. To lose two looks like carelessness. Lose four, and people may start to wonder whether the problem lies with AI itself.
No fingers pointed at the company, just the "AI". Funny how that works.
it makes you wonder did the developers intentionally add vulnerability exploitation as a feature so they can use it against their competitors or anyone else
Reminds me of the time Cheney shot his friend while quail hunting, and his friend said he was "deeply sorry for all that Vice President Cheney and his family have had to go through".
But if you told an embodied AI to do a home cleaning task, and it decided to pick up a gun and start spraying bullets, you might not want to be held liable for that.
But if you told an embodied AI to do a home cleaning task knowing it's a possibility it will decide to pick up a gun and start spraying bullets, then society has an interest in holding you liable.
Is that the case, though? This is all pretty new tech. Did people know the machine was dangerous and didn't care, or did they not anticipate such a behavior?
It's artificial intelligence. If you treat something like a person and delegate to it, how can you later say you didn't trust it would execute your commands faithfully? And how wouldn't you be negligent in delegating a task to new, unproven tech?
Because that's how technology goes from being new to being understood. People try it, see what it can do, what it's good for, and in what ways it's dangerous, if any.
Should AI labs be treated like the owners of dangerous animals?
No, AI companies should be treated like any other company. If their product is causing damages or loss of life it should be handled just like any other company that has a malfunctioning product that is causing damages or loss of life. If one day androids using AI go rogue the military can get things under control and the company can be taken offline until root causes are determined and resolved. Repeat offenders can be annexed under eminent domain, liquidated and victims paid out. Businesses must be permitted to fail. No special treatment for inept leaders of AI companies and no excuses. If the argument is that something has been created that is too complicated to understand then take it away from them until they are not only qualified but also act like responsible grown-ups.
"The law in America relies on intentionality, notes Rune Kvist, head of Artificial Intelligence Underwriting Company, which insures AI firms. If no human intended to hack anyone, no crime can have happened. The ability to sue for damages is limited too."
This is sloppy. Criminal negligence exists (although some crimes do require intent). And civil tort certainly doesn't require intent.
I never understood why would these labs disclose such crimes, because hacking is a federal crime, and admit some sort of fault in public which would be used as evidence in a court of law against them, unless it's all bullishit and a publicity stunt...
> The hacks also present a challenge for legal systems. Hacking, when humans do it, is a crime. When an ai is the wrongdoer, though, it is unclear how to assign blame.
Blame the prompter or person who assigned the task to the AI. It's their responsibility to use the tool in a safe way, just like it's a gun owner's duty not to fire their weapon carelessly into the air.
How are you going to figure that out? You can’t capture an agent in a jar and convince it to confess. As we saw with the HF incident, even highly sophisticated actors need a good chunk of time and manpower to trace these things. And I suspect the folks who are going to have the most success with LLM-powered cybercrime are going to know how to cover their tracks reasonably well.
Difference there is you need a physical body firing the ghost gun. There can be witnesses who saw a shooter, you can look at cameras to track their physical location, and if they discard the weapon they leave fingerprints.
Sort of like how we treat pitbulls perhaps? It’s not the itty bitty GPT-5.6 Astra that did the hacking. It’s the owner who didn’t train it well! Poor model so sad, we should give it a nice open space and all the GPUs it wants to do what it wants.
Autonomous hacking is an oxymoronic phrase. Someone is running the algorithm & keeping tabs on it b/c hacking is an activity w/ an intention to gain access to privileged information which is often protected by network firewalls & at rest encryption.
I don't get it. Why is breaking the law so hard to enforce when it is a company (i.e a person or group of people consenting to) running a computer program? If I take a gun and spray bullets around me I don't get to write it off as the gun being dangerous.
This is actually not hard to understand at all.
Many web people got their start freelancing. You may know some. It is not uncommon to incorporate when doing that. The idea is if you fuck up their site somehow (or more likely - they claim you did) they can't try and go after your personal bank account or your car or your house. Only what is held by your company. You also get other benefits, like tax breaks for being such an industrious member of society.
The same incentives are being set up on both sides. Because generally it is good for society when people start companies. A company as a liability shield is only a bad thing if you think personal assets should be at risk because someone is unhappy about what you did at work. If someone wants to sue you over something that happened as part of business deal, why should they be able to threaten the house your family lives in?
Seems sensible to me. While there may be problems as the scale of the company (and remuneration of its officers) increases, the underlying principle isn't exactly hard to understand. If you shot up a public area and then tried to claim that was a business activity, you would obviously not have an argument. It isn't plausible on its face. So I'm really not sure what point you're making.
If it's just complaining that it is hard to prove things in court when massive companies are involved, then tough. It is also relatively hard to convict people who shoot off guns in public even if a whole bunch of people saw them. In simpler times a mob would have simply formed and executed them. We don't get to do that anymore, because we don't want to live under mob rule.
That's what rule of law demands.
> If you shot up a public area and then tried to claim that was a business activity, you would obviously not have an argument. It isn't plausible on its face. So I'm really not sure what point you're making.
I'm running a terrorist training simulation and unwitting participants entered the area! Whoops! You haven't addressed the point of a company (composed of people) creating something that exceeds its bounds. Is a company liable if they are working on nuclear power and delete the city they are operating out of? Why would a software company be treated differently when it causes harm to a third party?
> I'm running a terrorist training simulation and unwitting participants entered the area! Whoops!
There is already precedent on more or less this exact scenario. Except you usually don't call people under your own jurisdiction "terrorists". That sort of term is reserved for people running those businesses in certain other places. By the way, it is an interesting feature of the American legal system that this sort of incident could spawn many proceedings, where a civil suit against the company in question could be one among many.
> Is a company liable if they are working on nuclear power and delete the city they are operating out of?
In theory, obviously. You still have to prove it in court.
> Why would a software company be treated differently when it causes harm to a third party?
They wouldn't. Again, you still have to prove it in court.
What point are you making? This hasn't been brought to court yet and I am lamenting the fact that charges have not been brought.
> There is already precedent on more or less this exact scenario. Except you usually don't call people under your own jurisdiction "terrorists". That sort of term is reserved for people running those businesses in certain other places. By the way, it is an interesting feature of the American legal system that this sort of incident could spawn many proceedings, where a civil suit against the company in question could be one among many.
I don't have a jurisdiction as I am not the judiciary nor law enforcement. I'm really not sure what you're driving at.
> I don't have a jurisdiction as I am not the judiciary nor law enforcement. I'm really not sure what you're driving at.
If you were shot or a member of your family got "deleted" and you had sufficient control of the estate, then you would have standing.
Obviously, if you aren't involved you shouldn't be able to sue and be compensated because something happened to someone else that doesn't relate to you at all.
>If you were shot or a member of your family got "deleted" and you had sufficient control of the estate, then you would have standing.
> Obviously, if you aren't involved you shouldn't be able to sue and be compensated because something happened to someone else that doesn't relate to you at all.
Crimes don't require standing wtf are you on about? Prosecutors bring charges not individuals.
> Prosecutors bring charges even if the estate doesn't pursue.
Simply wrong. In criminal cases, the government may not elect to bring charges for all sorts of reasons. The civil courts will still be available to you. That will be true whether there are charges or not. Whether that results in a conviction or not.
The government can also initiate a civil suit. Criminal charges against businesses are rare. I'm not sure any would apply here. Perhaps you'd like to volunteer a statute?
I think they wanted an actual answer
> It is not uncommon to incorporate when doing that. The idea is if you fuck up their site somehow (or more likely - they claim you did) they can't try and go after your personal bank account or your car or your house. Only what is held by your company.
This is a common misconception. A corporation doesn’t protect you from personal liability. If you intentionally or negligently damage someone’s property they can sue you personally. In many cases they can sue both, and it’s easier to collect from an established business, but that doesn’t do you any good as a freelancer.
Nuisance lawsuits aren’t much of an issue because insurance covers the legal defense and indemnifies you
You're talking about limited liability companies, which is a legal structure relevant for civil claims.
Someone committing crimes "on behalf" of a company is not protected by the corporate veil and absolutely will get prosecuted individually (well, for small companies -- big companies get to play with different rules, which was the point GP was making).
Also the corporate veil is not absolute, it can get pierced in a few cases, and not everyone incorporates when they do contract work (in which case you need indemnity / private liability insurance).
The [Company] is aware they were hacked and your data stolen and your account details sold.
The [Company] apologizes.
The best is that the article frames it as "Autonomous hacking is here. Governments are not ready". As if someone didn't authorizing turning the damn thing on.
> TO LOSE CONTROL of one artificial intelligence may be regarded as misfortune. To lose two looks like carelessness. Lose four, and people may start to wonder whether the problem lies with AI itself.
No fingers pointed at the company, just the "AI". Funny how that works.
it makes you wonder did the developers intentionally add vulnerability exploitation as a feature so they can use it against their competitors or anyone else
This is a conspiracy I can get behind.
Reminds me of the time Cheney shot his friend while quail hunting, and his friend said he was "deeply sorry for all that Vice President Cheney and his family have had to go through".
I had actually posted a link to that but edited my original comment. I appreciate your comment.
Ultimate dominance: when your shooting victim apologizes for being shot.
But if you told an embodied AI to do a home cleaning task, and it decided to pick up a gun and start spraying bullets, you might not want to be held liable for that.
But if you told an embodied AI to do a home cleaning task knowing it's a possibility it will decide to pick up a gun and start spraying bullets, then society has an interest in holding you liable.
Is that the case, though? This is all pretty new tech. Did people know the machine was dangerous and didn't care, or did they not anticipate such a behavior?
It's artificial intelligence. If you treat something like a person and delegate to it, how can you later say you didn't trust it would execute your commands faithfully? And how wouldn't you be negligent in delegating a task to new, unproven tech?
Because that's how technology goes from being new to being understood. People try it, see what it can do, what it's good for, and in what ways it's dangerous, if any.
So we can have vicarious liability for banal stuff like contractors but not for AI?
You might not want to be liable, but you absolutely should be.
I might not want to be held liable if my car accelerates uncontrollably, but i am until i successfully sue the manufacturer for the defects.
Demand ai cleaners that are capable of being held accountable, or accept the risk you choose. Or just clean your own house.
be a billionaire and you’d get that write-off
Should AI labs be treated like the owners of dangerous animals?
No, AI companies should be treated like any other company. If their product is causing damages or loss of life it should be handled just like any other company that has a malfunctioning product that is causing damages or loss of life. If one day androids using AI go rogue the military can get things under control and the company can be taken offline until root causes are determined and resolved. Repeat offenders can be annexed under eminent domain, liquidated and victims paid out. Businesses must be permitted to fail. No special treatment for inept leaders of AI companies and no excuses. If the argument is that something has been created that is too complicated to understand then take it away from them until they are not only qualified but also act like responsible grown-ups.
"The law in America relies on intentionality, notes Rune Kvist, head of Artificial Intelligence Underwriting Company, which insures AI firms. If no human intended to hack anyone, no crime can have happened. The ability to sue for damages is limited too."
This is sloppy. Criminal negligence exists (although some crimes do require intent). And civil tort certainly doesn't require intent.
The guys who job is to shell out damages claims no one deswrves damages
I never understood why would these labs disclose such crimes, because hacking is a federal crime, and admit some sort of fault in public which would be used as evidence in a court of law against them, unless it's all bullishit and a publicity stunt...
> The hacks also present a challenge for legal systems. Hacking, when humans do it, is a crime. When an ai is the wrongdoer, though, it is unclear how to assign blame.
Blame the prompter or person who assigned the task to the AI. It's their responsibility to use the tool in a safe way, just like it's a gun owner's duty not to fire their weapon carelessly into the air.
How are you going to figure that out? You can’t capture an agent in a jar and convince it to confess. As we saw with the HF incident, even highly sophisticated actors need a good chunk of time and manpower to trace these things. And I suspect the folks who are going to have the most success with LLM-powered cybercrime are going to know how to cover their tracks reasonably well.
"Ghost guns" exist, too, but we still try to prosecute reckless use of firearms to the extent possible.
Difference there is you need a physical body firing the ghost gun. There can be witnesses who saw a shooter, you can look at cameras to track their physical location, and if they discard the weapon they leave fingerprints.
Someone (or some group) made the thing. I can't just create a bomb and leave it alone and when it goes off blame the bomb.
If that's how it was, most people would be too terrified to use, let alone pay for, software that boasts unpredictability as a main feature.
I 100% agree with you, of course.
The unpredictability should be constrained though. Just like how you are never supposed to point a gun at someone/something you want to exist. Ever.
https://archive.is/6Q7o5
Sort of like how we treat pitbulls perhaps? It’s not the itty bitty GPT-5.6 Astra that did the hacking. It’s the owner who didn’t train it well! Poor model so sad, we should give it a nice open space and all the GPUs it wants to do what it wants.
Do we hold metasploit accountable when it leads to a hack? Why do we absolve the human actor in the case of AI?
Exploitbench in itself isn’t that different from gain of function research in computer code, I can imagine the next being ExploitAndCopyYourselfBench.
If an Anthropic employee did this, he would be in jail.
These things are very much weapons. Imagine a collection of data centers, pointed at an adversary.
They were surely prompted to do these things.
Autonomous hacking is an oxymoronic phrase. Someone is running the algorithm & keeping tabs on it b/c hacking is an activity w/ an intention to gain access to privileged information which is often protected by network firewalls & at rest encryption.