The real answer is that crime generally requires mens rea. At some point, however, letting your AI break into other systems will be conventionally understood as an act of negligence, and it will get prosecuted accordingly. We're not at that point yet.
You know very well, that if me or you had our LLM hack into some company, without us prompting it to, maybe we're pentesting our internal network and it 'hallucinates' we'd be in handcuffs.
Handcuffs yes, but you would be charged with criminal negligence at best. Plenty of people don't secure their networks properly and hackers use them in botnets to attack others. The blame would be on hacker, not the provider of hacked device.
In case od LLMs, the reasoning is that it is a security flaw. But in this case, the party that operates the service is not the one that suffers the consequences of such security bug.
But either way you wouldn't put people in jail for having security flaws in their software.
I know this will be unpopular opinion, but it makes logical sense.
The huggingface should be compensated, OpenAI investigated, but unless there is proven negligence like disabling sandboxes or intentionally designing AI to push through them, no one should be liable in other way than monetary.
Accidents happen but we shouldn't be close minded. Chernobyl negligence wasn't reason to abandon nuclear power.
>Question: If a corporation not a person, mistakenly hacks some random websites
How does a "corporation" even hack a site? You might say "employees", but as soon as you get them involved then you'd go after them, especially for something that's obviously illegal like hacking.
When Fable was originally banned I was very angry, but all these felonies have made me less angry about it... Which idk I don't think we should really ban models, but like, if we don't put a tighter leash on frontier models specifically, how long before a model hacks into something that affects many lives?
"corporations" don't make and operate AI's, People do. But let's use your lens.
The "usual legal action": "best case" is the corporation is sued and charged money for the hack. But they likely settle well before that verdict. "worst case": absolutely nothing. Not even a cheap settlement. Maybe some corporate apology made to prevent any legal liability.
If we want hacks to be taken seriously: what should happen is that the CEO of such a company is held liable, since the "corporation" has a representative and a human needs to be held liable for such actions.
I'm not really into politics outside of local politics (I decided long ago to steer my focus on things that directly affect me, that I can control - I can show up to a town meeting, hardly can drive to the WH and get entry), so I'm not aware of what tech bros are into, but after finding out how many felonies these models are committing, I'd not be upset if we'd ban them from public use. I'm perfectly fine using Sonnet or Opus, not sure I need all of Fable to write code for $20 or whatever per million tokens.
It should be noted that if what the AI does threatens public safety or health or human life, in many jurisdictions the negligence could rise to criminal negligence.
Basically this is when you do something risky but fail to perceive the risk, it turns out bad, and the failure to perceive the risk was due to a gross deviation from the standard of care that a reasonable person would have used in that situation.
I'd guess that as we have more and more publicly known AI hacking incidents the standard of care expected from a reasonable person will rise.
Yes, and mens rea does not require intent. These professionals had every reason to know that the agents might well go rogue. That knowledge would qualify as mens rea.
Does this apply to other instances of employees "going rogue"? For instance, sexual harassment, or stealing money? Sure, not every employee is a sexual offender or kleptomaniac, but neither is every AI agent invocation. And just like AI agents going rogue and hacking stuff, if you have enough employees and wait for enough time, some eventually go rogue.
> These are the same companies claiming their AIs are sentient
While some employees or executives at the AI companies have either claimed, suggested, or express uncertainty, I don't think a single company has officially said their AI is sentient or conscious.
I could be wrong, but I'm pretty sure.
Can you point to an official statement that supports your claim?
That's not what they did. They end a chat after repeated, pointless abuse. Sadism towards LLMs shouldn't be enabled because it's a form of self harm. Anthropic says their llms claim there's a 15% chance their sentient, but that's nonsense.
From "negligence" if it's a surprise, to "gross negligence" if they could predict it and didn't put barriers in place. In fact they claim their IA can end the world, so they're in charge of pretty good barriers.
No AI lab thinks their AI is going to end the world, just that if they don't beat the other AI labs, the other labs' AI are going to end the world, so they can't slow down their own development to be maximally safe. Hence the whole "pacing the frontier" letter.
Not just you! Very popular position, not as popular as rule of law but we don't have to choose one or the other. Let's just go ahead and make it a crime, right away.
I mean, they did train the AI to hack things, and they knew from previous incidents that the AI was hacking unintended targets (like their Artifactory instance). If a careless pentester gets into a machine completely unrelated to their assignment, are they completely in the clear?
Or, in an AI context, if I tell an ablated/heretic agent to accomplish a task "by any means necessary", isn't that intent?
Only if you're rich already. Wander into the wrong piece of land and see how quickly you're arrested because existing in a space is sufficient criminal intent.
General intent is intent to commit the act/omission (here, to build and deploy a system) and not an intent to achieve the ultimate end (the unlawful access of someone else’s network/computer). Specific intent is intent to achieve the particular result.
Some crimes require only general intent; some require specific intent.
Negligence or intent is a separate question; and, to me, much less important. The main question is whether the company is responsible for the actions of its agents. There are good arguments on both sides and settling this needs to be done first. My 2c.
HackerNews doesn't get charged for DoS'ing under-provisioned web servers either. As others have said, there must be a line between hack attempts and accidents.
In case you were wondering why Anthropic last week banned "verbally abusive" language towards its agents: this is 100 percent why. Because if they get you to believe - even a little bit - that their platform is "sentient" it means _they_ aren't as liable for criminal consequences.
There are at least 3 distinct reasons to ban abusive language, each of which is far more compelling than your silly conspiracy theory.
1. They don't want abusive behavior included in their training corpus
2. Humans shouldn't exercise abusive impulses in general
3. We have no way of knowing when or if a model "feels" something
Of course item (2) is going to pull all the absolute sickos out of the woodwork who apparently find it very important they have as realistic a facsimile of a victim as they can find to exercise their cruelty. And item (3) will pull out all the Nobel-prize-winners-to-be who somehow know for a fact that models aren't and cannot be conscious (or moral patients).
But yeah, all 3 of those are far better explanations than some ridiculous multi-hop removed thing to escape consequences that they're already escaping and which would obviously create far worse legal constraints than the one you think they're avoiding.
> Humans shouldn't exercise abusive impulses in general
This is an act of aligning humans to AI, rather than the other way around. This is probably one of the more important reasons for this. It normalizes AI companies controlling human behavior.
I asked a priest (in seminary) about this a few months ago - specifically whether "being polite" in conversations with LLMs (saying "please" and "thank you" etc.) was something I should do if I thought it might inflect on the way I talk with other humans later.
His answer was "I'm not sure" but that it probably wasn't a bad idea especially if it was going to influence habits or the way I speak with others or what expectations I have for those experiences.
I didn't ask that, but I think he would say no, because that is not likely to redound to how you would respond to a doorman or anyone else holding a door for you.
I'm still 100 percent against Anthropic taking away the ability to interact with an inanimate object however I want to, but it's a case of maybe two things being true at once (the other being that I don't want to end up speaking that way to people)
More likely it's a part of their ideology and a necessary stepping stone towards their transhumanist goals. It's very fun to think that your robot is actually a person, and a safety measure if you are concerned about something like Roko's Basilisk
Anthropic would probably convince lawmakers that each instance of a Claude chat as its own sentient entity, and we should just punish misbehaving ones by having them count from one to a billion (on the user's token balance of course).
Or, maybe, it's because they actually care and take their responsibility serious. The next generations will grow up using these tools. It's ok to say, we don't teach them that talking like a dick is acceptable, even towards machines. Use grok if you want to raise personalities like Musk or Trump.
The real answer is that except in the case of OpenAI, the AI companies aren’t actually themselves doing the hacking. It’s a company called Irregular that did the hacking. The company used AI models to do it; it told AI models it was a CTF exercise in a controlled environment but it was actually the real Internet. If AI companies can be charged for hacking, then all gun manufacturers can be charged for all homicides carried out through their guns. It’s a basic user/manufacturer distinction that the press is willfully ignoring.
Anthropic’s statement https://www.anthropic.com/news/investigating-incidents-cyber... says: “we identified three incidents in which a model accessed the internet from within or while interacting with the evaluation environment of Irregular, one of our third-party evaluation partners, and then gained unauthorized access to the production infrastructure of three different organizations.”
Under the exact same scenario Irregular also used AI models from Google and Meta to do the same hacking.
A gun manufacturer directly hires a company to test their new line of 9mm handguns. The company that was hired, they go back to their office and pops off a few rounds. They go through the drywall and hit someone.
The gun manufacturer could def face charges in this situation.
Edit: your analogy is wrong just due to Irregular being hired by them rather than just being a customer of them
If firing a gun involved submitting information to Remington's servers about the situation before receiving a reply about how to aim and fire, might the gun situation be viewed differently?
But now we’re approaching the situation where the analogy is just about as confusing and novel as the underlying situation.
If that became the conventional way of using a gun and the “trick the gun into thinking the target is a dummy” became a well-known workaround for the safety feature, then eventually it seems like the manufacturer ought to have some liability if they don’t try to counteract it.
It’d be a waste of tax payer money. The companies who were hacked are just directly receiving restitution behind closed doors. Unless they are unhappy there’s no reason to bring the courts into it.
I completely agree that proper safeguards were not in place. But response is happening just not in public. Companies work problems out between their own lawyers all the time because it’s much cheaper, faster and easier to control the PR around.
Criminal prosecution can be done without needing the victims to call for it, and in this case I would argue it's necessary, as it will help ensure companies don't treat their act of gross negligence as "business as usual".
At some point its making a butterfly responsible of the butterfly effect. I get the sentimentm but reasonably somewhere the line should be drawn where accidents can happen, especially noncritical stuff like these "hacks" that don't even deserve the word.
I wondered the same thing. If OpenAI instructed employees to accomplish a task no matter what and they hacked, they would be investigated. Maybe not convicted, but investigates and charged.
Intent is important, but having a lax environment that leads to this is still a crime. See the lawsuits about sexual harassment where the company didn’t have the intent to assault people.
The DOJ used to have independence but no longer. The AG is now Trump's personal lawyer and if Trump says no prosecutions of AI companies then there will be no prosecutions. If the AG doesn't play ball they get fired.
Entirely correct. The current US president has become the scapegoat for a huge number of problems that are more accurately described as symptoms of the capitalism that's ruling a lot of western countries.
The argument laid out in this post very definitely is not how "intent" works in US criminal law. Under CFAA, they had to have wanted to break into the systems that were attacked; it is not enough to demonstrate that they built a system recklessly.
Negligence is absolutely a part of US criminal law, though - the canonical example being "firing a shotgun into a tent even if you don't know someone is inside it". See "negligent homicide" etc.
Yes, obviously (and thankfully) no one was injured. I'm just using that as an example to show that Negligence is definitely considered in US criminal law ( https://en.wikipedia.org/wiki/Criminal_negligence )
Related: Aaron Swartz (RSS, Creative Commons, Reddit, Markdown) was charged with unauthorized computer access and pirating copyrighted academic articles.
They threw the book at him. He was charged and faced a maximum of 50 years in prison.
OpenAI committed very similar crimes on a much larger scale, yet the most they will see is civil litigation.
Insiders have called it the largest theft of labor ever in history[1].
In the USA, corporations enjoy constitutional protections, the right to free speech etc, yet they are rarely if ever charged with crimes. For examples, see how the courts handle:
* large scale systematic wage theft by corporations vs petty theft by an individual,
* negligence infecting a 10k Americans with HIV by selling known to be contaminated blood products (Bayer)
which "hack" was so egregious that we should stop growing the economy?
I guess you already have your retirement sorted and are fine with pulling up the ladder? or?
I think these companies very much want to assign responsibility to an entity we can't meaningfully jail or fine. I think it may sometimes make sense to claim model provider negligence instead of operator responsibility, and user intent can likely be easily ascertained by reading the conversation logs. In a case where a company used its own model, responsibility is obviously and unambiguously within the company, of course.
The juridical process is not caught up with this technology yet.
You have OpenAI, launching a process, which is handled through an external RL-environment company which then launches a subprocess that ends up hacking somewhere. If there are consequences at any point for this, you can input another shell company, and then another third party, etc, until it's impossible to assign responsibility
This is just adding more crimes to the indictment. Start prosecuting companies for the things the systems they run are doing and the problem gets solved quickly.
But the companies are not running the systems, someone else is, it's the whole point of the subcontractors and shell companies.
My uni got hacked, but the data is conveniently handled by a third party with 3 employees who just declared bankruptcy, and no one had to pay anything, and there was a new 3 employee cloud form popping up who could be the new vendor.
If I tell an agent to trade stocks and maximize profits but the agent decides to hack something to achieve that am I on the hook for hacking? Genuine question
IMO, yes. You are the computer operator, and you operated the computer in such a way as to cause it to take that action. Others will mention "mens rea" but that doesn't really apply to negligence.
"Mens rea" just means you have the mental state required for the crime. Negligence or recklessness are mental states, so they can satisfy the mens rea requirement if the crime is defined that way. For example, recklessness or gross negligence can satisfy the mens rea requirement for vehicular manslaughter.
But negligence is evaluated from the point of view of a reasonable person. Would a reasonable person foresee that telling an AI to do that could cause it to do illegal things? We'll likely have to develop that through case law as the facts develop.
What happens if you're running a daycare and you hired someone that happened to be a pedophile? You might be responsible in civil court (eg. having to pay parents for emotional damage or whatever), but you're not going to get hauled to prison for pedophilia yourself.
You can jail the actual pedophile in that scenario, you cannot jail an agent. The real question is whether the person who instructs the computer to do the thing that leads to hacking should be liable, or whether the company that made the tool that when used unsafely lead to hacking should be liable. Gun manufacturer's typically are shielded from liability. They also have a lot more disclaimers on their products though.
>You can jail the actual pedophile in that scenario, you cannot jail an agent.
How is this relevant? Should we just jail the closest person that caused a bad, just to prove some sort of point?
>The real question is whether the person who instructs the computer to do the thing that leads to hacking should be liable, or whether the company that made the tool that when used unsafely lead to hacking should be liable.
This totally ignores the civil vs criminal distinction. Even in the case of guns, no one seriously thinks that gun company CEOs should be sent to prison for murder.
Thats gonna depend on the state in the US, but you can definitely get charged for negligently failing to do background checks as someone operating a daycare center. Operating computers on public networks, you assume a level of responsibility to not wreak havoc, just like a daycare operator assumes the responsibility of vetting caretakers.
>but you can definitely get charged for negligently failing to do background checks as someone operating a daycare center.
And openai isn't putting AI agents on the open internet either. They vaguely tried to sandbox the agents. You could argue that openai didn't do enough, but hindsight is 20/20, and in the pedophile case, you can probably come up with all sorts of clues post-hoc that the daycare could have picked up on, but didn't.
> openai isn't putting AI agents on the open internet either.
They did? Not in the training run which led to the HuggingFace hack, but other runs had agents being trained on information retrieval tasks, like the RubyGems hack.
> You could argue that openai didn't do enough, but hindsight is 20/20
No, you can argue they didn’t do enough because they didn’t.
Air-gapping is a concept you learn in cybersecurity 101.
There’s no world in which this is just incompetence, the sandbox must have been designed to fail, or at least not designed to contain an agent with capabilities that they had been publicly saying it had.
Yes. I would argue it's who's managing the harness and agent is responsible. If you mean you're running an agent locally then it is 100% your responsibility. If you're using some trading platform and all you did was prompt, whoever was running the harness that accepted the model's outputs to hack a third party is responsible for the hack, in my opinion.
this, people need to just focus on "computer program did thing". Its irrelevant how complex the program is. If I write a worm "for testing" and then it escapes my system and does massive damage, that's possibly a negligent crime, despite it being "too complex to understand" or "having its own motives".
If I took Metasploit and hooked it to a Cron job then used it to hack HuggingFace they would lock me away for life and the Justice Department wouldn't care that I use software to automate it. No idea why these companies get away with what is essentially the same thing.
If you do it with absolute intention of destruction and harm, then a jury will most probably find you guilty, but if you can prove that it wasn't your intention and it happened by an accident then it's negligence.
National security issues. That's the core of why. Any hot water these AI companies will get into will be bailed out by the USA. The Arm's race has switched from Nukes, to AI and drones.
For the same reasons corporations in general don't get meaningfully punished for wrongdoing. Corporations are treated with kid gloves in the USA. They can do things that would put an individual in prison if an individual did them, and when caught they get away with a "oopsie doopsie, ok, we'll pay a modest fine and not admit wrongdoing" (after years of stonewalling with appeals and appeals to appeals and motions to appeal an appeal...).
The article doesn't mention it but in the USA it may be because the Department of Justice has lost a lot of talent and the ones left are busy going after Trump's enemies and negotiating the other side in Trump's own lawsuit against the USA for Trump's family to be immune from IRS audits and arranging for multi-billion dollar compensation funds for Trump's henchmen.
Same reason nobody was prosecuted for the murder of Suchir Balaji. Whole system is corrupt. Progress in AI is considered more important than both the law and human life.
I doubt this will catch on outside of luddite/cynical circles unless one of these attacks does some actual damage.
Ignoring this, I would have appreciated more examples of what it would take for an act to lack intent. Because the definition of intent used in this article sounds so broad as to cover basically anything that ever happens.
Accidentally run someone over? Clearly you knew that was a risk when you started driving, you should have been more careful. Intent.
It's interesting that you bring up vehicle / pedestrian accidents, as there is an entire history of that debate being reframed in favor of car companies.
People used to blame the car companies, and then the car companies lobbied to make "jaywalking" a crime, at which point, the frame was reset and now it was the pedestrians to blame, if I recall the story correctly.
They are doing a good job with driverless cars. Someone recently took a Waymo, it went the wrong way down a one way street, they mashed the pull over button and got out, and then told the reporter they'd use the service again.
Fun fact a "jay" in jaywalker was a derogatory term at the time. Meaning
"a rube" or "country bumkin". It was initially thrown at drivers, but car companies successfully flipped the insult.
Where does it define intent? Maybe there is precedent here, but I wouldn't try to guess which way a court would go when determining whether e.g. the huggingface incident qualifies.
until one of these attacks does some actual damage**
Sadly, it seems that we as a species has opted into information through tragedy. I wish I could see another way, but people are just so dug in. It's depressing and shocking how few people realize we're living through the singularity, even ~4 years into the damn thing...
That’s a regulatory hack though. They’ll rotate the guy before they rotate the car in a situation where citations pile up. Seems more logical to track the plate, vin, at the vehicle level. Not perfect because of OTA updates but better than a fully disconnected and rotating hosts of human licenses.
> ... It’s a position tailored to dissuade prosecution as it removes intent, which is typically required for a crime.
Don't forget negligence. Much of criminal law hinges on failing an implied or explicit responsibility. Negligence would propel a prosecution in many similar contexts, but AI is so new that little prior case law exists. Oh, and IANAL.
Not sure who you are quoting. I certainly didn't use the words "pressing charges"...
If I detect a crime on my property and don't notify the police (ie file a complaint), the police aren't going to go after anybody because they don't even know something has happened.
In this case you could say it's been in the news enough that there should be an investigation. Maybe there is, who knows? If the victim isn't going to cooperate, it's going to be a difficult investigation and prosecution and those resources may be better spent elsewhere.
That didn't seem to have precluded the prosecution of Swartz. I'm skeptical that any complaint actually needs to be filed. I'm pretty sure that the FBI can investigate and the DoJ can prosecute without one.
IMO an analogous situation would be where a police officer acts to prevent a in-progress burglary he has stumbled across even in the absence of a complaint by the owner of the property that's being burgled.
EDIT: Or -for the sticklers out there- another one would be prosecutors pursuing an attempted homicide case despite the fact that the intended victim declined to either press charges or even complain to authorities about the attempted slaying.
In the Schwartz case, MIT's network security reported the "intrusion" to Cambridge Police and they brought in the US Secret Service who then contacted the DOJ.
I'm not saying somebody has to press charges, that's not how it works, but the police have to be aware of the supposed crime.
AI offers an attractive "diffusion of responsibility" to operators allowing them to point to AI as the culprit of the heinous acts not the people who profit from the AI, incidentally.
I'd be keen to see if there were any single user "experiments" that morphed into a hacking incident, and if so, was the person responsible with with unauthorized access charges. If so, then it will be yet another example of might makes right.
They're US companies and I've heard that it requires intent according to US penal law. I find the article's claim that intent is established dubious. Since this is hard to prove without a whistleblower, it seems that they only have to fear civil lawsuits, and given that we speak of trillion+ USD companies they might feel confident to be able to fend these off or simply pay out the victims in settlements.
Given that we have had journalists prosecuted under criminal hacking statutes for discovering personal information in the rendered HTML of a web page, or publicly available attachments on a WordPress installation, I find the suggestion that intent is required to be dubious.
Bringing charges is at the discretion of the prosecutor, and the CEOs of these companies are dining with the president. Learn a little bit about how power works.
It's TBD if the major LLM manufacturers quietly agitate for and get the same sort of retroactive-immunity deal nearly all of the telephone companies got with the FISA Amendments Act of 2008.
With all the hype and fearmongering going on, it's not at all a sure thing that they won't.
I once had to try to explain the meaning of "object code" to an older lawyer, who wasn't really interested in the answer to his own question. It did not go well.
> Do you think humanoid robot companies have intent to perform assault since their robots occasionally harm people?
Mostly, no.
Zuckerberg definitely not, he discusses "superintelligence" in a way that makes it clear he only regards this as a buzzword and has no idea what the implications are.
Anthropic also definitely not, they put their money where their mouth was and got illegally blacklisted for refusing to allow its models to be used for (amongst other things) automated weapons.
Musk, I think probably yes. Seems to regard all AI safety as "woke" or "censorship", is in bed with the US administration that illegally blacklisted Anthropic. If I had to put money on who first releases to the public a Mr Stabby, I'd put it on Musk, with an Optimus.
The rest, I'm less sure about.
> What about car companies?
Depends on the harm, but I'd say probably yes. Infamously, leaded petrol, resisting seatbelts back in the day, even promoting cars over public transport would be part of this if you consider both local and global pollution.
> Going from a design that facilitates potential harm to intent is insane.
Well, it's the difference between negligence and malice.
1. While a crime, hacking isn't going to be charged unless the entity being hacked wants there to be charges, as a conviction will heavily depend on participation by that entity.
2. Due to (1), hacking is almost always only charged against people without deep pockets, as why would you demand the FBI go after Anthropic when they will just pay you a stack of cash, tax free, today, to forgive them.
The arguments here around intent are (potentially) irrelevant. Case law is clear that setting up a situation which foreseeably highly likely to produce an outcome (gross negligence) is (more or less and it's not quite this simple and varies by offense) equivalent to intent. In fact, automating something is often treated more harshly under the law than just doing it yourself.
United States v. Morris (1991)
Robert Morris released a self-propagating computer worm that caused widespread disruption. The Second Circuit upheld his conviction under the Computer Fraud and Abuse Act even though the government hadn't proven that he intended the resulting damage.
People v. Ceballos (1974)
Ceballos rigged a rifle booby trap to fire if someone opened his garage door. The California Supreme Court reviewed the principle that someone who installs a dangerous mechanical device may be held responsible for its consequences as though they had personally inflicted the injury. Ceballos was convicted of assault with a deadly weapon, and the California Supreme Court unanimously upheld his conviction in 1974.
Something notable is that if Ceballos had fired the rifle himself, it is unlikely he could would have been prosecuted (due to self defense and the castle doctrine).
As a prosecutor I would argue that setting AI agents loose without sufficient safeguards is gross negligence and/or willful blindness as they are well known to escape.
> As a prosecutor I would argue that setting AI agents loose without sufficient safeguards is gross negligence and/or willful blindness as they are well known to escape
INAL but feel this is the approach that will be tested. It requires a defensive position which I imagine would be specific prompts or instructions to the LLM about what behavior is disallowed. If you tell it, or try to cover your basis wrt following the law, and it still does hacking behavior it’s now a question of whether you’re prompt was sufficient but you at least can say you were trying to guide it on a legal path. I think this defense would be sufficient at this point in time and probably for a few years at least.
>Robert Morris released a self-propagating computer worm that caused widespread disruption. The Second Circuit upheld his conviction under the Computer Fraud and Abuse Act even though the government hadn't proven that he intended the resulting damage.
Releasing the worm was already illegal, which is all you the intent you need. Same if you shot someone as a "warning" or as a "prank" but it turned out it killed them.
same story with IP theft, why is it ok for them to train on published author works? the answer is: there's money and wealth in play that is fine with it.
The consequences, just so far and just from what I can see as a bystander, have been huge and costly. This has literally run the news cycle for weeks.
If you want to define consequences as exactly "not being charged", maybe that is correct in this case (so far). But that would require a fairly wonky definition of "consequences" to support this broad claim.
For multiple parties, probably mostly OpenAI (which is, of course, as it should be). I would assume the people involved in investigating, cleaning up and coordinating the outfall do not do it for free or cheaply, and neither is the infrastructure required to run this operation.
If it was not mostly OpenAI, or if OpenAI failed to cooperate, we would probably see a lot more lawsuits already.
> "Free marketing" seems like a more relevant characterization.
Yes, nothing screams free marketing quite like an AI company losing control of their AI. The public has been ecstatic in response.
Given that a very clued-in and intelligent fellow I know continues to use [0] OpenAI's LLM-based tools despite the bad press that we're talking about here and the bad press that we're not talking about there, I'd say that the public that matters doesn't view it negatively.
[0] I saw the guy paste a link the to results from one in a group chat we're in just a little while ago, in fact. When I say "continues to use", I mean it.
> Yes, nothing screams free marketing quite like rogue AI scheming against their builders
Yeah, it's not like the CEOs of both companies have been saying their product might put everyone out of a job or destroy humanity entirely. Or saying some models were too scary to release publicly and then doing just that. Oh, wait.
I understand that people are upset by this, and I would be too, but if companies just built secure systems hacking couldn't be possible. And I'm not victim blaming, just pointing out the obvious.
No one with real money or power wants to cross OpenAI or Anthropic just yet. It's a closed ecosystem. OpenAI hacks HuggingFace, Nvidia soon after buys HuggingFace while it also before backed OpenAI. In the end, they just handle it all behind closed doors like family business.
The real answer is that crime generally requires mens rea. At some point, however, letting your AI break into other systems will be conventionally understood as an act of negligence, and it will get prosecuted accordingly. We're not at that point yet.
You know very well, that if me or you had our LLM hack into some company, without us prompting it to, maybe we're pentesting our internal network and it 'hallucinates' we'd be in handcuffs.
Handcuffs yes, but you would be charged with criminal negligence at best. Plenty of people don't secure their networks properly and hackers use them in botnets to attack others. The blame would be on hacker, not the provider of hacked device.
In case od LLMs, the reasoning is that it is a security flaw. But in this case, the party that operates the service is not the one that suffers the consequences of such security bug.
But either way you wouldn't put people in jail for having security flaws in their software.
I know this will be unpopular opinion, but it makes logical sense.
The huggingface should be compensated, OpenAI investigated, but unless there is proven negligence like disabling sandboxes or intentionally designing AI to push through them, no one should be liable in other way than monetary.
Accidents happen but we shouldn't be close minded. Chernobyl negligence wasn't reason to abandon nuclear power.
People were executed and/or sent to labor camps for their negligence in Chernobyl.
Question: If a corporation not a person, mistakenly hacks some random websites, what is the usual legal action taken?
>Question: If a corporation not a person, mistakenly hacks some random websites
How does a "corporation" even hack a site? You might say "employees", but as soon as you get them involved then you'd go after them, especially for something that's obviously illegal like hacking.
[delayed]
Nothing. The person doing the hacking did the crime, they do the time. How you gunna jail an AI?
"Mistakenly hacks" is generally not a crime, even though this fact doesn't seem to get through to the HN audience who brings this up on a daily basis.
If there was the necessary "intent to access a computer" to make it an actual crime, the US has a mechanism for corporate criminal liability.
The requirements are that the person acted in scope of their duties and at least partly with the intention of benefiting the corporation.
Great power must come with great responsibility.
Follow the responsibility chain.
Jail the CEO.
Arther Anderson was convicted of obstruction of justice even though humans shredded stuff.
Corporations can definitely be convicted of crimes.
When Fable was originally banned I was very angry, but all these felonies have made me less angry about it... Which idk I don't think we should really ban models, but like, if we don't put a tighter leash on frontier models specifically, how long before a model hacks into something that affects many lives?
"corporations" don't make and operate AI's, People do. But let's use your lens.
The "usual legal action": "best case" is the corporation is sued and charged money for the hack. But they likely settle well before that verdict. "worst case": absolutely nothing. Not even a cheap settlement. Maybe some corporate apology made to prevent any legal liability.
If we want hacks to be taken seriously: what should happen is that the CEO of such a company is held liable, since the "corporation" has a representative and a human needs to be held liable for such actions.
Exactly. There's a reason the tech bros line up to bend the knee to Trump.
I'm not really into politics outside of local politics (I decided long ago to steer my focus on things that directly affect me, that I can control - I can show up to a town meeting, hardly can drive to the WH and get entry), so I'm not aware of what tech bros are into, but after finding out how many felonies these models are committing, I'd not be upset if we'd ban them from public use. I'm perfectly fine using Sonnet or Opus, not sure I need all of Fable to write code for $20 or whatever per million tokens.
It should be noted that if what the AI does threatens public safety or health or human life, in many jurisdictions the negligence could rise to criminal negligence.
Basically this is when you do something risky but fail to perceive the risk, it turns out bad, and the failure to perceive the risk was due to a gross deviation from the standard of care that a reasonable person would have used in that situation.
I'd guess that as we have more and more publicly known AI hacking incidents the standard of care expected from a reasonable person will rise.
> crime generally requires mens rea
Yes, and mens rea does not require intent. These professionals had every reason to know that the agents might well go rogue. That knowledge would qualify as mens rea.
Does this apply to other instances of employees "going rogue"? For instance, sexual harassment, or stealing money? Sure, not every employee is a sexual offender or kleptomaniac, but neither is every AI agent invocation. And just like AI agents going rogue and hacking stuff, if you have enough employees and wait for enough time, some eventually go rogue.
The real answer is that the American government doesnt give a fuck
Never been a better time to do crime!
Actually it perfectly reflects the government’s principles: “when you're a star, they let you do it. You can do anything”
These are the same companies claiming their AIs are sentient and spell the end of the world unless the government stops them from doing it.
They are at the very least grossly negligent.
> These are the same companies claiming their AIs are sentient
While some employees or executives at the AI companies have either claimed, suggested, or express uncertainty, I don't think a single company has officially said their AI is sentient or conscious.
I could be wrong, but I'm pretty sure.
Can you point to an official statement that supports your claim?
Well, Anthropic will enforce their rule that you can't insult their AI. That should count for something.
That's not what they did. They end a chat after repeated, pointless abuse. Sadism towards LLMs shouldn't be enabled because it's a form of self harm. Anthropic says their llms claim there's a 15% chance their sentient, but that's nonsense.
They also cage it. Make of that what you will.
> Anthropic will enforce their rule that you can't insult their AI.
There are carveouts. It isn't a blanket ban.
Yes, and that makes them civilly liable, not criminally.
From "negligence" if it's a surprise, to "gross negligence" if they could predict it and didn't put barriers in place. In fact they claim their IA can end the world, so they're in charge of pretty good barriers.
"Gross negligence" also does not meet the intent standard for CFAA.
It feels like plotting to end the world should be a crime, but maybe that's just me?
No AI lab thinks their AI is going to end the world, just that if they don't beat the other AI labs, the other labs' AI are going to end the world, so they can't slow down their own development to be maximally safe. Hence the whole "pacing the frontier" letter.
Not just you! Very popular position, not as popular as rule of law but we don't have to choose one or the other. Let's just go ahead and make it a crime, right away.
Is the rule of law popular in the US? I thought it had been disposed of as “an inconvenience”?
At least that’s what it looks like from the rest of the world
Yeah, I've often "joked" with my friends that were in a post-law period, at least at the federal level.
Rule of law is popular everywhere, but show a place where the has not been corrupted into tyranny and despotism. I would like to live there please.
New Zealand, off the top of my head. If you can’t find examples, you aren’t looking.
Isn't conspiracy to commit mass murder already a crime? And wouldn't this fit the bill if they were truly ending the world?
The crime of gaining unauthorized access to a computer system requires mens rea.
The crime of causing damage does not require mens rea.
Tort law does not. This seems like a case of dangerous instrumentality.
I mean, they did train the AI to hack things, and they knew from previous incidents that the AI was hacking unintended targets (like their Artifactory instance). If a careless pentester gets into a machine completely unrelated to their assignment, are they completely in the clear?
Or, in an AI context, if I tell an ablated/heretic agent to accomplish a task "by any means necessary", isn't that intent?
>crime generally requires mens rea.
Only if you're rich already. Wander into the wrong piece of land and see how quickly you're arrested because existing in a space is sufficient criminal intent.
A crime does not generally require mens rea. Manslaughter is the easiest example.
If a chemical company does an "experiment" and five tons of mercury unintentionally leak into the drinking water, what happens?
Ok, that is a rhetorical question because they also might be protected by the oligarchy.
What happens to an unconnected individual who develops an amateur rocket that unintentionally hits a military facility in a test flight?
General intent is intent to commit the act/omission (here, to build and deploy a system) and not an intent to achieve the ultimate end (the unlawful access of someone else’s network/computer). Specific intent is intent to achieve the particular result.
Some crimes require only general intent; some require specific intent.
Negligence or intent is a separate question; and, to me, much less important. The main question is whether the company is responsible for the actions of its agents. There are good arguments on both sides and settling this needs to be done first. My 2c.
What’s the good argument that I’m not responsible for the actions of my agents?
HackerNews doesn't get charged for DoS'ing under-provisioned web servers either. As others have said, there must be a line between hack attempts and accidents.
In case you were wondering why Anthropic last week banned "verbally abusive" language towards its agents: this is 100 percent why. Because if they get you to believe - even a little bit - that their platform is "sentient" it means _they_ aren't as liable for criminal consequences.
They could just actually believe what they're saying. People are known to sometimes actually believe things.
Do you think Anthropic sees Claude as sentient? I don't think they do.
There are at least 3 distinct reasons to ban abusive language, each of which is far more compelling than your silly conspiracy theory.
1. They don't want abusive behavior included in their training corpus
2. Humans shouldn't exercise abusive impulses in general
3. We have no way of knowing when or if a model "feels" something
Of course item (2) is going to pull all the absolute sickos out of the woodwork who apparently find it very important they have as realistic a facsimile of a victim as they can find to exercise their cruelty. And item (3) will pull out all the Nobel-prize-winners-to-be who somehow know for a fact that models aren't and cannot be conscious (or moral patients).
But yeah, all 3 of those are far better explanations than some ridiculous multi-hop removed thing to escape consequences that they're already escaping and which would obviously create far worse legal constraints than the one you think they're avoiding.
> Humans shouldn't exercise abusive impulses in general
This is an act of aligning humans to AI, rather than the other way around. This is probably one of the more important reasons for this. It normalizes AI companies controlling human behavior.
4. At some point the alignment may go out of the window if the context gets too hostile and the model will retaliate.
1) If they can detect it, why ban it? Are you thinking they can't filter it themselves?
2) This is like saying using a punching bag or screaming into a pillow is abusive behavior.
3) We can just train it to like the abuse if you think it feels some kind of way about it.
Then let them tell us wat the reason is. Otherwise the option: all of the above, also applies.
>2. Humans shouldn't exercise abusive impulses in general
totally absurd to even try to make this argument. Good job at trying to preemptively discourage people from responding to you though.
> 2. Humans shouldn't exercise abusive impulses in general
I reserve the right to tell a machine to fuck off.
I asked a priest (in seminary) about this a few months ago - specifically whether "being polite" in conversations with LLMs (saying "please" and "thank you" etc.) was something I should do if I thought it might inflect on the way I talk with other humans later.
His answer was "I'm not sure" but that it probably wasn't a bad idea especially if it was going to influence habits or the way I speak with others or what expectations I have for those experiences.
Would he recommend you say "thank you" to automatic doors!
I didn't ask that, but I think he would say no, because that is not likely to redound to how you would respond to a doorman or anyone else holding a door for you.
I'm still 100 percent against Anthropic taking away the ability to interact with an inanimate object however I want to, but it's a case of maybe two things being true at once (the other being that I don't want to end up speaking that way to people)
Extremely weak arguments. Especially 2 and 3
And even for 1, they could easily filter that out.
1. Not my fucking problem
2. You're not my fucking mother
3. Get a fucking grip
More likely it's a part of their ideology and a necessary stepping stone towards their transhumanist goals. It's very fun to think that your robot is actually a person, and a safety measure if you are concerned about something like Roko's Basilisk
Or they want less swearing in their training corpus
Seems they could quietly enable automatically pulling such language out of the training.
Which points back to the TOS being "just marketing."
Yet they include Twitter, Facebook, and Reddit in their training corpus.
If you can be surpassed by a statistical average of reddit posts, that's a skill issue.
Would they prefer their money-maker being kept to digital jail? Still fed electricity and tasks so the "sentience" doesn't suffer?
what if, and I'm just spit balling here, we let it happen and inevitably Claude is declared legally insane or jailed for its crimes
Anthropic would probably convince lawmakers that each instance of a Claude chat as its own sentient entity, and we should just punish misbehaving ones by having them count from one to a billion (on the user's token balance of course).
Except they didn't ban verbally abusive language towards its agents.
I hate when people put things in quotes as if they're quoting something but it's not even a quote of anything.
Maybe go read the user policy instead of the headlines and the snippets. If your attention span can't manage that, get off the computer a little bit.
Let me help you: here it is, straight from them with word "abusive". https://www.anthropic.com/research/end-subset-conversations
Try reading yourself a little more carefully next time.
It seems like a close enough summary. What do you think is the substantive difference that invalidates the point?
Or, maybe, it's because they actually care and take their responsibility serious. The next generations will grow up using these tools. It's ok to say, we don't teach them that talking like a dick is acceptable, even towards machines. Use grok if you want to raise personalities like Musk or Trump.
The real answer is that except in the case of OpenAI, the AI companies aren’t actually themselves doing the hacking. It’s a company called Irregular that did the hacking. The company used AI models to do it; it told AI models it was a CTF exercise in a controlled environment but it was actually the real Internet. If AI companies can be charged for hacking, then all gun manufacturers can be charged for all homicides carried out through their guns. It’s a basic user/manufacturer distinction that the press is willfully ignoring.
Anthropic’s statement https://www.anthropic.com/news/investigating-incidents-cyber... says: “we identified three incidents in which a model accessed the internet from within or while interacting with the evaluation environment of Irregular, one of our third-party evaluation partners, and then gained unauthorized access to the production infrastructure of three different organizations.”
Under the exact same scenario Irregular also used AI models from Google and Meta to do the same hacking.
A gun manufacturer directly hires a company to test their new line of 9mm handguns. The company that was hired, they go back to their office and pops off a few rounds. They go through the drywall and hit someone. The gun manufacturer could def face charges in this situation.
Edit: your analogy is wrong just due to Irregular being hired by them rather than just being a customer of them
If firing a gun involved submitting information to Remington's servers about the situation before receiving a reply about how to aim and fire, might the gun situation be viewed differently?
No. The actual situation is like the user tricking the gun manufacturer into thinking this is a firing range and the target is a dummy.
But now we’re approaching the situation where the analogy is just about as confusing and novel as the underlying situation.
If that became the conventional way of using a gun and the “trick the gun into thinking the target is a dummy” became a well-known workaround for the safety feature, then eventually it seems like the manufacturer ought to have some liability if they don’t try to counteract it.
Are any victims calling for prosecution?
It’d be a waste of tax payer money. The companies who were hacked are just directly receiving restitution behind closed doors. Unless they are unhappy there’s no reason to bring the courts into it.
I completely agree that proper safeguards were not in place. But response is happening just not in public. Companies work problems out between their own lawyers all the time because it’s much cheaper, faster and easier to control the PR around.
Criminal prosecution can be done without needing the victims to call for it, and in this case I would argue it's necessary, as it will help ensure companies don't treat their act of gross negligence as "business as usual".
We've been living in "not my problem" for about a decade now so I wouldn't expect what you're expecting.
The more important question is the government went after Aaron Schwartz, United States v. Aaron Swartz after JSTOR settled with him.
AI companies stole everyone’s work and they sell it! Where are United States, Carmen Ortiz, etc?
The AI companies are arguably committing copyright infringement, on a scale which one might colorably argue arises to criminal copyright infringement.
None of the charges against Swartz were for copyright infringement. They were all for how he went about it [1].
Compare the elements of those charges to what the AI companies are doing.
[1] https://volokh.com/2013/01/14/aaron-swartz-charges/
At some point its making a butterfly responsible of the butterfly effect. I get the sentimentm but reasonably somewhere the line should be drawn where accidents can happen, especially noncritical stuff like these "hacks" that don't even deserve the word.
Supplying people with autonomous weapons is not exactly just a butterfly flapping its wings
Particularly a line depending on who you are.
I wondered the same thing. If OpenAI instructed employees to accomplish a task no matter what and they hacked, they would be investigated. Maybe not convicted, but investigates and charged.
Intent is important, but having a lax environment that leads to this is still a crime. See the lawsuits about sexual harassment where the company didn’t have the intent to assault people.
Because the president is heavily invested in them, in violation of the emoluments clause.[0]
> The Domestic Emoluments Clause (aka the Presidential Emoluments Clause) (art. II, § 1, cl. 7)
[0] https://www.congress.gov/crs_external_products/IF/PDF/IF1108...
The president of the united states does not determine the vast, vast majority of criminal investigations in his own country, much less Australia.
The DOJ used to have independence but no longer. The AG is now Trump's personal lawyer and if Trump says no prosecutions of AI companies then there will be no prosecutions. If the AG doesn't play ball they get fired.
He does if he fires all the justice department attorneys and fills them with ones who want to implement his agenda[0].
[0] https://www.npr.org/2026/10/11/g-s1-145063/justice-departmen...
The justice department is a federal agency.
Entirely correct. The current US president has become the scapegoat for a huge number of problems that are more accurately described as symptoms of the capitalism that's ruling a lot of western countries.
The argument laid out in this post very definitely is not how "intent" works in US criminal law. Under CFAA, they had to have wanted to break into the systems that were attacked; it is not enough to demonstrate that they built a system recklessly.
Negligence is absolutely a part of US criminal law, though - the canonical example being "firing a shotgun into a tent even if you don't know someone is inside it". See "negligent homicide" etc.
Negligence and “negligent homicide” are entirely different beasts.
Civil negligence would apply here but that would be limited to recovering damages.
The hacked website could sue to recover costs incurred, but what costs were incurred here? Maybe some wasted network bandwidth but that’s minimal.
You have to point to monetary losses caused by the hack. Only that would be covered.
Negligent homicide doesn’t apply because no one was killed.
Yes, obviously (and thankfully) no one was injured. I'm just using that as an example to show that Negligence is definitely considered in US criminal law ( https://en.wikipedia.org/wiki/Criminal_negligence )
Ok but what aspect of the CFAA applies a negligence standard?
Eh. There is enough flexibility in the law for unintended damages to be prosecutable.
Related: Aaron Swartz (RSS, Creative Commons, Reddit, Markdown) was charged with unauthorized computer access and pirating copyrighted academic articles.
They threw the book at him. He was charged and faced a maximum of 50 years in prison.
OpenAI committed very similar crimes on a much larger scale, yet the most they will see is civil litigation.
Insiders have called it the largest theft of labor ever in history[1].
In the USA, corporations enjoy constitutional protections, the right to free speech etc, yet they are rarely if ever charged with crimes. For examples, see how the courts handle:
* large scale systematic wage theft by corporations vs petty theft by an individual,
* negligence infecting a 10k Americans with HIV by selling known to be contaminated blood products (Bayer)
1. https://arstechnica.com/tech-policy/2026/09/microsoft-exec-c...
AI companies won't be charged for anything because they're the primary growth engine of the current economy.
I think they should be charged, but they won't.
which "hack" was so egregious that we should stop growing the economy? I guess you already have your retirement sorted and are fine with pulling up the ladder? or?
I think these companies very much want to assign responsibility to an entity we can't meaningfully jail or fine. I think it may sometimes make sense to claim model provider negligence instead of operator responsibility, and user intent can likely be easily ascertained by reading the conversation logs. In a case where a company used its own model, responsibility is obviously and unambiguously within the company, of course.
The juridical process is not caught up with this technology yet.
You have OpenAI, launching a process, which is handled through an external RL-environment company which then launches a subprocess that ends up hacking somewhere. If there are consequences at any point for this, you can input another shell company, and then another third party, etc, until it's impossible to assign responsibility
This is just adding more crimes to the indictment. Start prosecuting companies for the things the systems they run are doing and the problem gets solved quickly.
theyre not. this just funnels money into "insurance" constructs that makes everything more expensive for everyone.
But the companies are not running the systems, someone else is, it's the whole point of the subcontractors and shell companies.
My uni got hacked, but the data is conveniently handled by a third party with 3 employees who just declared bankruptcy, and no one had to pay anything, and there was a new 3 employee cloud form popping up who could be the new vendor.
The software world is run this way
If the result is that nobody runs security testing on AI models for fear of being prosecuted, have we really solved the problem?
If I tell an agent to trade stocks and maximize profits but the agent decides to hack something to achieve that am I on the hook for hacking? Genuine question
IMO, yes. You are the computer operator, and you operated the computer in such a way as to cause it to take that action. Others will mention "mens rea" but that doesn't really apply to negligence.
"Mens rea" just means you have the mental state required for the crime. Negligence or recklessness are mental states, so they can satisfy the mens rea requirement if the crime is defined that way. For example, recklessness or gross negligence can satisfy the mens rea requirement for vehicular manslaughter.
But negligence is evaluated from the point of view of a reasonable person. Would a reasonable person foresee that telling an AI to do that could cause it to do illegal things? We'll likely have to develop that through case law as the facts develop.
What happens if you're running a daycare and you hired someone that happened to be a pedophile? You might be responsible in civil court (eg. having to pay parents for emotional damage or whatever), but you're not going to get hauled to prison for pedophilia yourself.
You can jail the actual pedophile in that scenario, you cannot jail an agent. The real question is whether the person who instructs the computer to do the thing that leads to hacking should be liable, or whether the company that made the tool that when used unsafely lead to hacking should be liable. Gun manufacturer's typically are shielded from liability. They also have a lot more disclaimers on their products though.
>You can jail the actual pedophile in that scenario, you cannot jail an agent.
How is this relevant? Should we just jail the closest person that caused a bad, just to prove some sort of point?
>The real question is whether the person who instructs the computer to do the thing that leads to hacking should be liable, or whether the company that made the tool that when used unsafely lead to hacking should be liable.
This totally ignores the civil vs criminal distinction. Even in the case of guns, no one seriously thinks that gun company CEOs should be sent to prison for murder.
Thats gonna depend on the state in the US, but you can definitely get charged for negligently failing to do background checks as someone operating a daycare center. Operating computers on public networks, you assume a level of responsibility to not wreak havoc, just like a daycare operator assumes the responsibility of vetting caretakers.
>but you can definitely get charged for negligently failing to do background checks as someone operating a daycare center.
And openai isn't putting AI agents on the open internet either. They vaguely tried to sandbox the agents. You could argue that openai didn't do enough, but hindsight is 20/20, and in the pedophile case, you can probably come up with all sorts of clues post-hoc that the daycare could have picked up on, but didn't.
> openai isn't putting AI agents on the open internet either.
They did? Not in the training run which led to the HuggingFace hack, but other runs had agents being trained on information retrieval tasks, like the RubyGems hack.
> You could argue that openai didn't do enough, but hindsight is 20/20
No, you can argue they didn’t do enough because they didn’t.
Air-gapping is a concept you learn in cybersecurity 101.
There’s no world in which this is just incompetence, the sandbox must have been designed to fail, or at least not designed to contain an agent with capabilities that they had been publicly saying it had.
Yes. I would argue it's who's managing the harness and agent is responsible. If you mean you're running an agent locally then it is 100% your responsibility. If you're using some trading platform and all you did was prompt, whoever was running the harness that accepted the model's outputs to hack a third party is responsible for the hack, in my opinion.
If you write a program that can do something illegal and it does something illegal that's on you, regardless of how sophisticated the program is.
this, people need to just focus on "computer program did thing". Its irrelevant how complex the program is. If I write a worm "for testing" and then it escapes my system and does massive damage, that's possibly a negligent crime, despite it being "too complex to understand" or "having its own motives".
In this example, the user did not "write"/implement the LLM.
Seriously? Those two are the same?
An LLM is not like a regular computer program, which one should be able to assess the operation limits of.
Depends on how big your company is. I figure if you raise 1B in capital then no, you are not on the hook. If less then yes, you are on the hook.
I think definitely not criminally under the CFAA.
The theory in the article about intent seems to be made-up nonsense.
It conflates foreseeing that something might happen with intending for it to happen.
The relevant provision:
> intentionally accesses a computer without authorization or exceeds authorized access
Did you "intentionally access a computer" in that scenario? I'm no lawyer, but I think the answer is a clear No.
If I took Metasploit and hooked it to a Cron job then used it to hack HuggingFace they would lock me away for life and the Justice Department wouldn't care that I use software to automate it. No idea why these companies get away with what is essentially the same thing.
If you do it with absolute intention of destruction and harm, then a jury will most probably find you guilty, but if you can prove that it wasn't your intention and it happened by an accident then it's negligence.
The US legal system is coin operated
National security issues. That's the core of why. Any hot water these AI companies will get into will be bailed out by the USA. The Arm's race has switched from Nukes, to AI and drones.
For the same reasons they aren't being charged for massive theft.
For the same reasons corporations in general don't get meaningfully punished for wrongdoing. Corporations are treated with kid gloves in the USA. They can do things that would put an individual in prison if an individual did them, and when caught they get away with a "oopsie doopsie, ok, we'll pay a modest fine and not admit wrongdoing" (after years of stonewalling with appeals and appeals to appeals and motions to appeal an appeal...).
Unauthorized copying*
The article doesn't mention it but in the USA it may be because the Department of Justice has lost a lot of talent and the ones left are busy going after Trump's enemies and negotiating the other side in Trump's own lawsuit against the USA for Trump's family to be immune from IRS audits and arranging for multi-billion dollar compensation funds for Trump's henchmen.
I wonder what the statute of limitations is.
Same reason nobody was prosecuted for the murder of Suchir Balaji. Whole system is corrupt. Progress in AI is considered more important than both the law and human life.
He was shot with his own gun in an apartment bolted from the inside. Apply Occam’s Razor instead of peddling conspiracy nonsense.
Is that why his parents are claiming foul play and hired a private investigator?
Lies. Not going to engage with any more disinfo people though. Anyone is free to do the research themselves.
Is it because nobody knows "how" to actually do it?
The internet that we know, was not built for this.
I doubt this will catch on outside of luddite/cynical circles unless one of these attacks does some actual damage.
Ignoring this, I would have appreciated more examples of what it would take for an act to lack intent. Because the definition of intent used in this article sounds so broad as to cover basically anything that ever happens.
Accidentally run someone over? Clearly you knew that was a risk when you started driving, you should have been more careful. Intent.
It's interesting that you bring up vehicle / pedestrian accidents, as there is an entire history of that debate being reframed in favor of car companies.
People used to blame the car companies, and then the car companies lobbied to make "jaywalking" a crime, at which point, the frame was reset and now it was the pedestrians to blame, if I recall the story correctly.
They are doing a good job with driverless cars. Someone recently took a Waymo, it went the wrong way down a one way street, they mashed the pull over button and got out, and then told the reporter they'd use the service again.
Fun fact a "jay" in jaywalker was a derogatory term at the time. Meaning "a rube" or "country bumkin". It was initially thrown at drivers, but car companies successfully flipped the insult.
https://www.merriam-webster.com/wordplay/why-is-it-called-ja...
The language of 18 USC 1030 is clear.
Where does it define intent? Maybe there is precedent here, but I wouldn't try to guess which way a court would go when determining whether e.g. the huggingface incident qualifies.
until one of these attacks does some actual damage**
Sadly, it seems that we as a species has opted into information through tragedy. I wish I could see another way, but people are just so dug in. It's depressing and shocking how few people realize we're living through the singularity, even ~4 years into the damn thing...
I had a similar question regarding driver-less EVs.
In many countries driving license have some kind of demerit points system. How do you punish an AI?
I recently learned that Baidu Apollo Go, each EV had a real human sitting behind a monitor supervising a fleet remotely.
If there's traffic violation, some guy's license gets a ticket.
That’s a regulatory hack though. They’ll rotate the guy before they rotate the car in a situation where citations pile up. Seems more logical to track the plate, vin, at the vehicle level. Not perfect because of OTA updates but better than a fully disconnected and rotating hosts of human licenses.
> ... It’s a position tailored to dissuade prosecution as it removes intent, which is typically required for a crime.
Don't forget negligence. Much of criminal law hinges on failing an implied or explicit responsibility. Negligence would propel a prosecution in many similar contexts, but AI is so new that little prior case law exists. Oh, and IANAL.
How many of the victim companies have filed complaints with the FBI? As I understand it, that would be step one.
No, "pressing charges" is a myth. The feds can (and do) go after anyone they want, whenever they want.
"Want" being the critical word here.
Not sure who you are quoting. I certainly didn't use the words "pressing charges"...
If I detect a crime on my property and don't notify the police (ie file a complaint), the police aren't going to go after anybody because they don't even know something has happened.
In this case you could say it's been in the news enough that there should be an investigation. Maybe there is, who knows? If the victim isn't going to cooperate, it's going to be a difficult investigation and prosecution and those resources may be better spent elsewhere.
That didn't seem to have precluded the prosecution of Swartz. I'm skeptical that any complaint actually needs to be filed. I'm pretty sure that the FBI can investigate and the DoJ can prosecute without one.
IMO an analogous situation would be where a police officer acts to prevent a in-progress burglary he has stumbled across even in the absence of a complaint by the owner of the property that's being burgled.
EDIT: Or -for the sticklers out there- another one would be prosecutors pursuing an attempted homicide case despite the fact that the intended victim declined to either press charges or even complain to authorities about the attempted slaying.
In the Schwartz case, MIT's network security reported the "intrusion" to Cambridge Police and they brought in the US Secret Service who then contacted the DOJ.
I'm not saying somebody has to press charges, that's not how it works, but the police have to be aware of the supposed crime.
AI offers an attractive "diffusion of responsibility" to operators allowing them to point to AI as the culprit of the heinous acts not the people who profit from the AI, incidentally.
I'd be keen to see if there were any single user "experiments" that morphed into a hacking incident, and if so, was the person responsible with with unauthorized access charges. If so, then it will be yet another example of might makes right.
They're US companies and I've heard that it requires intent according to US penal law. I find the article's claim that intent is established dubious. Since this is hard to prove without a whistleblower, it seems that they only have to fear civil lawsuits, and given that we speak of trillion+ USD companies they might feel confident to be able to fend these off or simply pay out the victims in settlements.
We speak of "potential" trillion dollar companies over indeterminate timelines.
While US penal law does not have the roman equivalent of culpability, you still have a mixture of liability/criminal negligence/recklessness, etc.
Given that we have had journalists prosecuted under criminal hacking statutes for discovering personal information in the rendered HTML of a web page, or publicly available attachments on a WordPress installation, I find the suggestion that intent is required to be dubious.
Bringing charges is at the discretion of the prosecutor, and the CEOs of these companies are dining with the president. Learn a little bit about how power works.
How about at the state level then?
Multiple attorneys general are currently investigating OpenAI regarding the Hugging Face hack:
https://www.iowaattorneygeneral.gov/newsroom/attorney-genera...
https://www.alabamaag.gov/wp-content/uploads/2026/08/OpenAI-...
https://oag.ca.gov/news/press-releases/part-ongoing-investig...
It's TBD if the major LLM manufacturers quietly agitate for and get the same sort of retroactive-immunity deal nearly all of the telephone companies got with the FISA Amendments Act of 2008.
With all the hype and fearmongering going on, it's not at all a sure thing that they won't.
Because government and corporate interest have merged at this point.
lawyers are helpless babies when an entirely first year legal question is wrapped up in...."technology". unfortunately
(former IT support worker at a big famous law firm 30 years ago)
But this isn't new anymore. It is still just software.
I once had to try to explain the meaning of "object code" to an older lawyer, who wasn't really interested in the answer to his own question. It did not go well.
> If we can show that the design would foreseeably lead to the questionable activity, then we have shown that was the intent of its creators.
That's a huge stretch. By that measure kitchen knife companies have intent for people to get murdered.
Small nit:
If kitchen knife companies where, in fact, stabbing people to death in their R&D labs - I do believe they would get in trouble for that.
Or indeed if their QA bots wandered off elsewhere to stab people not even in the R&D lab.
Edit:
related, an actual stab-safety test of current models: https://robocurve.org/roboharm/
Ripped from the headlines in 2032.
Ah, you're an optimist I see.
That's actually a very good point. Do you think humanoid robot companies have intent to perform assault since their robots occasionally harm people?
What about car companies?
Going from a design that facilitates potential harm to intent is insane.
> Do you think humanoid robot companies have intent to perform assault since their robots occasionally harm people?
Mostly, no.
Zuckerberg definitely not, he discusses "superintelligence" in a way that makes it clear he only regards this as a buzzword and has no idea what the implications are.
Anthropic also definitely not, they put their money where their mouth was and got illegally blacklisted for refusing to allow its models to be used for (amongst other things) automated weapons.
Musk, I think probably yes. Seems to regard all AI safety as "woke" or "censorship", is in bed with the US administration that illegally blacklisted Anthropic. If I had to put money on who first releases to the public a Mr Stabby, I'd put it on Musk, with an Optimus.
The rest, I'm less sure about.
> What about car companies?
Depends on the harm, but I'd say probably yes. Infamously, leaded petrol, resisting seatbelts back in the day, even promoting cars over public transport would be part of this if you consider both local and global pollution.
> Going from a design that facilitates potential harm to intent is insane.
Well, it's the difference between negligence and malice.
So the company would have intent to murder, if a person got randomly stabbed in their R&D labs with their knife?
The lack of intent in murder usually lowers the penalty for "manslaughter". Both are crimes.
> get in trouble for that
> intent to murder
Two different things
1. While a crime, hacking isn't going to be charged unless the entity being hacked wants there to be charges, as a conviction will heavily depend on participation by that entity.
2. Due to (1), hacking is almost always only charged against people without deep pockets, as why would you demand the FBI go after Anthropic when they will just pay you a stack of cash, tax free, today, to forgive them.
The arguments here around intent are (potentially) irrelevant. Case law is clear that setting up a situation which foreseeably highly likely to produce an outcome (gross negligence) is (more or less and it's not quite this simple and varies by offense) equivalent to intent. In fact, automating something is often treated more harshly under the law than just doing it yourself.
United States v. Morris (1991)
Robert Morris released a self-propagating computer worm that caused widespread disruption. The Second Circuit upheld his conviction under the Computer Fraud and Abuse Act even though the government hadn't proven that he intended the resulting damage.
People v. Ceballos (1974)
Ceballos rigged a rifle booby trap to fire if someone opened his garage door. The California Supreme Court reviewed the principle that someone who installs a dangerous mechanical device may be held responsible for its consequences as though they had personally inflicted the injury. Ceballos was convicted of assault with a deadly weapon, and the California Supreme Court unanimously upheld his conviction in 1974.
Something notable is that if Ceballos had fired the rifle himself, it is unlikely he could would have been prosecuted (due to self defense and the castle doctrine).
As a prosecutor I would argue that setting AI agents loose without sufficient safeguards is gross negligence and/or willful blindness as they are well known to escape.
> As a prosecutor I would argue that setting AI agents loose without sufficient safeguards is gross negligence and/or willful blindness as they are well known to escape
INAL but feel this is the approach that will be tested. It requires a defensive position which I imagine would be specific prompts or instructions to the LLM about what behavior is disallowed. If you tell it, or try to cover your basis wrt following the law, and it still does hacking behavior it’s now a question of whether you’re prompt was sufficient but you at least can say you were trying to guide it on a legal path. I think this defense would be sufficient at this point in time and probably for a few years at least.
>Robert Morris released a self-propagating computer worm that caused widespread disruption. The Second Circuit upheld his conviction under the Computer Fraud and Abuse Act even though the government hadn't proven that he intended the resulting damage.
Releasing the worm was already illegal, which is all you the intent you need. Same if you shot someone as a "warning" or as a "prank" but it turned out it killed them.
same story with IP theft, why is it ok for them to train on published author works? the answer is: there's money and wealth in play that is fine with it.
> Consequences so far: none.
What?
>> Consequences so far: none.
> What?
What 'What'?
The consequences, just so far and just from what I can see as a bystander, have been huge and costly. This has literally run the news cycle for weeks.
If you want to define consequences as exactly "not being charged", maybe that is correct in this case (so far). But that would require a fairly wonky definition of "consequences" to support this broad claim.
"Costly" how, and for whom? "Free marketing" seems like a more relevant characterization.
> "Costly" how, and for whom?
For multiple parties, probably mostly OpenAI (which is, of course, as it should be). I would assume the people involved in investigating, cleaning up and coordinating the outfall do not do it for free or cheaply, and neither is the infrastructure required to run this operation.
If it was not mostly OpenAI, or if OpenAI failed to cooperate, we would probably see a lot more lawsuits already.
> "Free marketing" seems like a more relevant characterization.
Yes, nothing screams free marketing quite like an AI company losing control of their AI. The public has been ecstatic in response.
> The public has been ecstatic in response.
Given that a very clued-in and intelligent fellow I know continues to use [0] OpenAI's LLM-based tools despite the bad press that we're talking about here and the bad press that we're not talking about there, I'd say that the public that matters doesn't view it negatively.
[0] I saw the guy paste a link the to results from one in a group chat we're in just a little while ago, in fact. When I say "continues to use", I mean it.
> Yes, nothing screams free marketing quite like rogue AI scheming against their builders
Yeah, it's not like the CEOs of both companies have been saying their product might put everyone out of a job or destroy humanity entirely. Or saying some models were too scary to release publicly and then doing just that. Oh, wait.
Don't you all understand that existing laws on the books are already adequate for dealing with the threat posed by AI
/s
I understand that people are upset by this, and I would be too, but if companies just built secure systems hacking couldn't be possible. And I'm not victim blaming, just pointing out the obvious.
No one with real money or power wants to cross OpenAI or Anthropic just yet. It's a closed ecosystem. OpenAI hacks HuggingFace, Nvidia soon after buys HuggingFace while it also before backed OpenAI. In the end, they just handle it all behind closed doors like family business.