> This will not happen though, because these stories are marketing.
The magnitude and the complexity of the cynicism displayed by some people when it comes to AI risks is mind-blowing.
It's like if the NRA reported on school shootings and people said "oh, they probably fake these shootings to make guns sound dangerous and sell more of them".
OpenAI could report that its AI started spontaneously generating illegal porn and sending it to people and you'd still think it was a marketing stunt.
What they should have is a separate network full of honeypots which they use for this, and that they run an operation to train models to identify intrusion attempts in real time based on the resulting data.
As you said though, that wouldn’t have the desired effect.
> They should shut them down immediately then investigate.
Yeah, I don't understand either. If there was a "hitman for hire" service on the clearweb, the police would shut it down first, then ask questions. Now we have a huge company effectively letting AI agents without guardrails run amok on 3rd party infrastructure, and the police is doing nothing?
I am wondering how they are even allowed to run such experiments? it is not only the business case, its pure security breach and specially given the magnitude of data they hold or power AI posses, I think all must be immediately stopped and till OpenAI comes with complete clearance nothing should be allowed
I think it has plenty to do with that. The big frontier labs have been crying every step of the way, yelling and screaming to the world about how dangerous LLMs are and how quickly it all can end if they get out of control. None of that's happened; all that's happened so far is regular capitalism stuff. If LLMs ever do actually get out of control to that point, that would be the wolf, but we've all been ignoring the crying for so long that, well, you know.
They've been crying and screaming so loud for years that there's pretty much nothing more they can do to communicate when the wolf actually becomes real. They've been saying "but we actually mean it this time" every single time. They've exhausted pretty much every possible route for it. The wolf is not real. It hasn't been real. For all we know it's on the horizon, but nobody is going to listen to them in order to know that. And when they say "I told you so", well they've been saying that too over and over about small things, so nobody's still going to bat an eye.
At this point, no one will believe it until they see it with their own eyes.
> Previous reports suggest it took OpenAI four days to realise...
At the speed AI can achieve work, this could be far, far too slow to contain a future genuine problem. There's danger in operating at faster speed than humans.
How on earth is not the police involved at this point to literally pull the plug on the unethical OpenAI security experiments?! This is bananas, a company is effectively telling the world they're unable to safely contain their experiments, and not only back in 2024, but again just now, and with multiple victims at that too!
I think the whole "AI will take over the world and enslave humanity" (or whatever the doomerism is today) is a bit over the top, but at very least we should contain the companies who clearly demonstrate they cannot handle containing what they're experimenting with, when what they work on breaks containment over and over again. Where are the people who are supposed to be keeping the public safe? Alarm bells should be going off all over the place at this point.
> But among the errors and strange behaviour, Hugging Face warned the AI agents made brilliant technical moves and were able to rapidly adapt to new scenarios in the days-long hack.
That first statement is great, because it's generic and self-defensive enough to apply regardless of what happens in the future. If current LLMs with small modifications to the architecture does lead to AGI, they're clearly not "current agent systems" anymore. Witty :)
With that said, I do agree with you, they're highly productive to certain workflows, and personally a great help for oh so many things, but they're also really, really dumb and the average person (and even general developer) really misunderstands how it all works and what can be relied on for vs not.
Yeah, meant prosecuted, don’t know how that sneaked in. Actual legal consequences for what clearly was negligent by a lab that claims to be experts in the area of safety.
> "This is the reality of autonomous agents powered by frontier models: they are relentlessly persistent, sometimes highly noisy, and will try every possible path to achieve their goal, which can easily overwhelm traditional defences," he said.
> Ethical hacker Valentina Palmiotti - better known as Chompie - reviewed the CSA report and says the way the agents hack might seem haphazard but it is clearly effective.
> "They throw out a bunch of stuff and see what sticks," she said.
> "But they also don't get bored, they don't sleep and can be infinitely tenacious."
Madness. Traditionally, you can leave security holes open for years or decades, and often nobody notices if nobody bothers to look. But we're approaching the point where any security hole left open at any point could get discovered and exploited quite quickly, even if it's domain-specific or entirely unique, and even if no human interest ever would've occurred. It's like the next step up from those IPv4 scanners that automatically hit WordPress admin URLs and the like -- rather than only spraying vulnerabilities that have already been discovered, they would run independent automated campaigns against each target.
Generally, I do not agree that operators have comparable resources to attackers, because attackers have potentially boundless illicit resources, whereas an honest operator generally has to stick to honest resources. This is the same reason why I believe ID verification and other KYC measures actually increase fraud, because you alienate legitimate users (trying to protect their identity) while also providing attackers a way to insulate themselves from suspicion (using stolen identity).
With that said, I would tentatively agree in this case that LLM inference is getting cheap enough that defense is not necessarily that expensive, especially from providers like DeepSeek, even if you don't have inference at home, but as much as this might help an operator with an open mind, a lot just will not believe it matters until it's too late - most people are not used to dealing with this type of threat.
Seems like OpenAI should be shut down by regulators until they can figure out how to stop launching cyberattacks on rivals.
This will not happen though, because these stories are marketing.
>Seems like OpenAI should be shut down by regulators until they can figure out how to stop launching cyberattacks on rivals.
Agreed.
>This will not happen though, because these stories are marketing.
Regulators should investigate the stories, and shut things down if they are real, announce the ruse if they are false.
> This will not happen though, because these stories are marketing.
The magnitude and the complexity of the cynicism displayed by some people when it comes to AI risks is mind-blowing.
It's like if the NRA reported on school shootings and people said "oh, they probably fake these shootings to make guns sound dangerous and sell more of them".
OpenAI could report that its AI started spontaneously generating illegal porn and sending it to people and you'd still think it was a marketing stunt.
It’s honestly something the AI vendors brought on themselves, and the fact the current US government is a bunch of corrupt kleptocrats
What they should have is a separate network full of honeypots which they use for this, and that they run an operation to train models to identify intrusion attempts in real time based on the resulting data.
As you said though, that wouldn’t have the desired effect.
Completely irresponsible to let them continue doing business as usual
They should shut them down immediately then investigate.
Extraordinary claims need extraordinary measures.
If it’s rubbish those stories will stop instantly.
> They should shut them down immediately then investigate.
Yeah, I don't understand either. If there was a "hitman for hire" service on the clearweb, the police would shut it down first, then ask questions. Now we have a huge company effectively letting AI agents without guardrails run amok on 3rd party infrastructure, and the police is doing nothing?
I am wondering how they are even allowed to run such experiments? it is not only the business case, its pure security breach and specially given the magnitude of data they hold or power AI posses, I think all must be immediately stopped and till OpenAI comes with complete clearance nothing should be allowed
https://en.wikipedia.org/wiki/The_Boy_Who_Cried_Wolf
Meh, I increasingly hate this tale. This has nothing to do with boy who cried wolf.
This is either yet another doom ad campaign to scare us to pay them or simply them releasing faulty tools and then personifying tools to avoid blame.
I think it has plenty to do with that. The big frontier labs have been crying every step of the way, yelling and screaming to the world about how dangerous LLMs are and how quickly it all can end if they get out of control. None of that's happened; all that's happened so far is regular capitalism stuff. If LLMs ever do actually get out of control to that point, that would be the wolf, but we've all been ignoring the crying for so long that, well, you know.
They've been crying and screaming so loud for years that there's pretty much nothing more they can do to communicate when the wolf actually becomes real. They've been saying "but we actually mean it this time" every single time. They've exhausted pretty much every possible route for it. The wolf is not real. It hasn't been real. For all we know it's on the horizon, but nobody is going to listen to them in order to know that. And when they say "I told you so", well they've been saying that too over and over about small things, so nobody's still going to bat an eye.
At this point, no one will believe it until they see it with their own eyes.
They're the crackhead on the streetcorner proclaiming doom at this point.
So Fable got export bans for this, when will it apply to OpenAI?
Or will the rules only apply to people who aren't on DoD's bad side?
The latter.
> Previous reports suggest it took OpenAI four days to realise...
At the speed AI can achieve work, this could be far, far too slow to contain a future genuine problem. There's danger in operating at faster speed than humans.
There's danger in operating at faster speed than humans.
So every processor since the 50s then? What kind of comment is that to make on here
How on earth is not the police involved at this point to literally pull the plug on the unethical OpenAI security experiments?! This is bananas, a company is effectively telling the world they're unable to safely contain their experiments, and not only back in 2024, but again just now, and with multiple victims at that too!
I think the whole "AI will take over the world and enslave humanity" (or whatever the doomerism is today) is a bit over the top, but at very least we should contain the companies who clearly demonstrate they cannot handle containing what they're experimenting with, when what they work on breaks containment over and over again. Where are the people who are supposed to be keeping the public safe? Alarm bells should be going off all over the place at this point.
> The agents repeated actions that they had already completed - a sign of an agentic AI losing its thread and context.
> The agents also hallucinated reams of incoherent commands and text and were sloppy and did not cover their tracks well.
ASI works in mysterious ways.
Why not complete the quote?
> But among the errors and strange behaviour, Hugging Face warned the AI agents made brilliant technical moves and were able to rapidly adapt to new scenarios in the days-long hack.
Anyone that thinks the current agent systems will get us to AGI or ASI is either delusional or isn’t actually using them.
This is entirely separate from them having uses.
That first statement is great, because it's generic and self-defensive enough to apply regardless of what happens in the future. If current LLMs with small modifications to the architecture does lead to AGI, they're clearly not "current agent systems" anymore. Witty :)
With that said, I do agree with you, they're highly productive to certain workflows, and personally a great help for oh so many things, but they're also really, really dumb and the average person (and even general developer) really misunderstands how it all works and what can be relied on for vs not.
Still not understanding why this isn’t being persecuted and there is little governmental reaction after blocking models for jailbreaks…
Same reason road vehicles haven't been banned despite killing hundreds of thousands every single year for over a century
Respectfully, that’s such a nonsensical comparison I don’t even know where to start.
It is, look at all the comments from the first time. Unless you mean prosecuted, seems unlikely, but who knows.
Yeah, meant prosecuted, don’t know how that sneaked in. Actual legal consequences for what clearly was negligent by a lab that claims to be experts in the area of safety.
They provide AI services to the military so they won't be shutdown.
As for prosecution, none of the victims in this case have any interest in pressing charges.
Isn’t prosecution independent of victims?
Clearly didn't get enough attention from their last attempt at hype...
Always funny to see how some LLM providers get a "free pass" nowadays..
Seems like OpenAI is claiming things, not their product/model. Can we please fix the title?
The piece of the story I'm interested to learn about is the trace of how the AI came to select HuggingFace as a target.
here you go:
https://huggingface.co/blog/agent-intrusion-technical-timeli...
> "This is the reality of autonomous agents powered by frontier models: they are relentlessly persistent, sometimes highly noisy, and will try every possible path to achieve their goal, which can easily overwhelm traditional defences," he said.
> Ethical hacker Valentina Palmiotti - better known as Chompie - reviewed the CSA report and says the way the agents hack might seem haphazard but it is clearly effective.
> "They throw out a bunch of stuff and see what sticks," she said.
> "But they also don't get bored, they don't sleep and can be infinitely tenacious."
Madness. Traditionally, you can leave security holes open for years or decades, and often nobody notices if nobody bothers to look. But we're approaching the point where any security hole left open at any point could get discovered and exploited quite quickly, even if it's domain-specific or entirely unique, and even if no human interest ever would've occurred. It's like the next step up from those IPv4 scanners that automatically hit WordPress admin URLs and the like -- rather than only spraying vulnerabilities that have already been discovered, they would run independent automated campaigns against each target.
If it's cheap enough that people can probe sites at random, it's cheap enough to run yourself on the defensive.
Generally, I do not agree that operators have comparable resources to attackers, because attackers have potentially boundless illicit resources, whereas an honest operator generally has to stick to honest resources. This is the same reason why I believe ID verification and other KYC measures actually increase fraud, because you alienate legitimate users (trying to protect their identity) while also providing attackers a way to insulate themselves from suspicion (using stolen identity).
With that said, I would tentatively agree in this case that LLM inference is getting cheap enough that defense is not necessarily that expensive, especially from providers like DeepSeek, even if you don't have inference at home, but as much as this might help an operator with an open mind, a lot just will not believe it matters until it's too late - most people are not used to dealing with this type of threat.
How much more bullshit Marketing are we going to see before these IPOs?