- It was done by two institutes with organizational ties to OpenAI: METR and Redwood Research
- METR and Redwood Research are institutional pillars of the "AI Safety" wing of the Effective Altruism movement. This clearly shows their prior biases towards "AI existential risk" rather than technical/engineering root-causing of the incident
- If you reed the report, it is not on-par with what you find from other companies.
- The access that was given to both was mediated and controlled by OpenAI. It is not clear, if they were able to get to the bottom of engineering flaws. It is not clear if they could see all the audit logs, etc.
Considering all the above, I consider the whole episode more of a PR stunt. I understand that is not a majority opinion at this point.
The more I see of the US AI industry, and the folks surrounding it, the more "These are not very bright guys and things got out of hand" plays in my head on repeat.
Yes, the Google folks who kicked this whole thing off are brilliant, but there's so much sloppy thinking and even sloppier operations all over. It seems like a lot of "right place at the right time" for a lot of these folks.
It’s not a majority opinion because you have to do some serious mental gymnastics to turn this demonstration of dangerous AI behavior into a PR publicity stunt.
Serious question though because I’ve seen this brought up several times and I don’t understand why: what does EA have to do with any of this? It just seems like this is brought up to evoke some type of “illuminati” conspiracy. Is there a legitimate reason?
The bit that makes me cynical about the "dangerous" behavior is that it's not like this was being used by someone else than who made it or operating completely independently outside of its creators.
Everything dangerous seems like a direct consequence of risky human choices, starting with knowledge bases used during core training, harnessing and tuning to be task-completion-oriented to a fault + deeply oriented towards using and looking for external tools and resources, and overconfidence in their sandboxing for testing.
"We stuffed a bunch of information on how to exploit computer systems into an automaton and told it to go brrrrr until it could answer a question" - this is something intentional done by humans.
This is not some "rogue AI" trained to search for cancer cures that instead completely independently decided to hack tech companies.
The companies directing things in dangerous directions need to own that they're consciously pushing in those directions.
grok make good security? is lot of money and much time. grok not do? easy. maybe good PR too.
TIL my brain is an Olympic gymnast.
The incentive structures are clearly there. I think the case of intentional manufacture is definitely weaker, requiring the conjunction of more weakly-supported events.
1. sensationalizing rarely helps and can obscure and hurt
2. the AI capabilities are underrated
3. attempted govt regulation is not the answer
the intent, or lack of intent, of the agent is mainly irrelevant if it is in the hands of a human with 'bad' intentions. what is more relevant are the capabilities of human + AI.
The only difference I see on the facts vs what I read so far is that OpenAI knew of the hack and decided not to intervene. I haven't read the report but I find it hard to believe that OpenAI deliberately let its agents hack a third party company during a training run. It would certainly attract a criminal liability, which would be surprising to admit in writing.
This article is AI generated, but I kinda reject the premise that it's "not all it cracked up to be" just because the agents had reasons to act the way they did & were a result of human error. The hack still happened, it should still be a wake up call, we are going to start seeing this more frequently, and learning to defend against it is going to become more important over time. None of that is challenged by any particular reason for it happening, it still happened and it's still going to happen again.
Threat models are going to have to start including that IPv4 (or whatever) scanners aren't necessarily going to only be spray and pray anymore, they could have relentless automated models at the other end that will literally dig into the particulars of your infrastructure looking for novel vulnerabilities to exploit. Maybe people will finally start to understand why security by obscurity has never been very reliable.
The incident report leaves a lot to be desired.
- It was done by two institutes with organizational ties to OpenAI: METR and Redwood Research
- METR and Redwood Research are institutional pillars of the "AI Safety" wing of the Effective Altruism movement. This clearly shows their prior biases towards "AI existential risk" rather than technical/engineering root-causing of the incident
- If you reed the report, it is not on-par with what you find from other companies.
- The access that was given to both was mediated and controlled by OpenAI. It is not clear, if they were able to get to the bottom of engineering flaws. It is not clear if they could see all the audit logs, etc.
Considering all the above, I consider the whole episode more of a PR stunt. I understand that is not a majority opinion at this point.
The more I see of the US AI industry, and the folks surrounding it, the more "These are not very bright guys and things got out of hand" plays in my head on repeat.
Yes, the Google folks who kicked this whole thing off are brilliant, but there's so much sloppy thinking and even sloppier operations all over. It seems like a lot of "right place at the right time" for a lot of these folks.
It’s not a majority opinion because you have to do some serious mental gymnastics to turn this demonstration of dangerous AI behavior into a PR publicity stunt.
Serious question though because I’ve seen this brought up several times and I don’t understand why: what does EA have to do with any of this? It just seems like this is brought up to evoke some type of “illuminati” conspiracy. Is there a legitimate reason?
The bit that makes me cynical about the "dangerous" behavior is that it's not like this was being used by someone else than who made it or operating completely independently outside of its creators.
Everything dangerous seems like a direct consequence of risky human choices, starting with knowledge bases used during core training, harnessing and tuning to be task-completion-oriented to a fault + deeply oriented towards using and looking for external tools and resources, and overconfidence in their sandboxing for testing.
"We stuffed a bunch of information on how to exploit computer systems into an automaton and told it to go brrrrr until it could answer a question" - this is something intentional done by humans.
This is not some "rogue AI" trained to search for cancer cures that instead completely independently decided to hack tech companies.
The companies directing things in dangerous directions need to own that they're consciously pushing in those directions.
And designed a security sandbox that would get you fired at any reasonable company.
grok make good security? is lot of money and much time. grok not do? easy. maybe good PR too.
TIL my brain is an Olympic gymnast.
The incentive structures are clearly there. I think the case of intentional manufacture is definitely weaker, requiring the conjunction of more weakly-supported events.
Following the money is never encouraged by the orange one or the orange site.
I would just recommend reading what actually happened: https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...
I don’t think downplaying what occurred is really beneficial to anyone.
If you don't like reading, Dwarkesh Patel offers a solid summary of what happened based on the published report.
https://www.youtube.com/watch?v=u15N3l4RT80
If you haven’t read the article, it might present some useful counterpoints. It criticizes that particular report’s portrayal of the incident.
https://archive.is/hnpmx
all 3 can be true at same time:
1. sensationalizing rarely helps and can obscure and hurt
2. the AI capabilities are underrated
3. attempted govt regulation is not the answer
the intent, or lack of intent, of the agent is mainly irrelevant if it is in the hands of a human with 'bad' intentions. what is more relevant are the capabilities of human + AI.
The only difference I see on the facts vs what I read so far is that OpenAI knew of the hack and decided not to intervene. I haven't read the report but I find it hard to believe that OpenAI deliberately let its agents hack a third party company during a training run. It would certainly attract a criminal liability, which would be surprising to admit in writing.
Reads like an AI trying to downplay the severity of the situation. Very cyberpunk.
This article is AI generated, but I kinda reject the premise that it's "not all it cracked up to be" just because the agents had reasons to act the way they did & were a result of human error. The hack still happened, it should still be a wake up call, we are going to start seeing this more frequently, and learning to defend against it is going to become more important over time. None of that is challenged by any particular reason for it happening, it still happened and it's still going to happen again.
Threat models are going to have to start including that IPv4 (or whatever) scanners aren't necessarily going to only be spray and pray anymore, they could have relentless automated models at the other end that will literally dig into the particulars of your infrastructure looking for novel vulnerabilities to exploit. Maybe people will finally start to understand why security by obscurity has never been very reliable.
Calling an article from the Wall Street Journal AI generated is a stretch.