> "The tool itself worked properly and functioned as intended; however due to a bug in a separate code path, the system did not properly verify that the email address provided by the individual requesting a password reset matched the email address associated with that user’s Instagram account," said Meta in its breach notice.
I'm not sure "worked properly" and "as intended" accurately describe this situation.
I also can't believe the people who were involved with writing this response from Meta, didn't realize how obviously bad it sounds. It's like there is no humans working and writing there anymore.
To be fair, that quote in the original article could have more context. By "The tool" they meant "AI-assisted support tool"[1]; perhaps they meant that the issue was not an AI hallucination inherent of the tool, but a fixable bug.
In that case, the statement is so meaningless as to be useless. Why should we care how Meta splits up their microservices? The tool still failed. They just want to redefine the "tool" as something else, anything else, to avoid having to admit something negative about our precious AI.
> The LLM correctly generated tokens according to user input, however due to a bug in a separate code path, the system did not properly verify the email address
> Nginx correctly handled the user requests according to the HTTP standard, however due to a bug in a separate code path, the system did not properly verify the email address
It probably could have been, but how likely is that compared to with the AI agent? I'd assume (and I'm ready to look like an idiot if I'm wrong) that the humans are trained to send the verification code to the email address on file, rather than any address the client asks them to. I'd certainly assume most of them are more afraid of the consequences than the AI is.
I get the joke, but it's a relevant nuance that the new code, the chatbot, did not have 'the bug'. I still think that the mistake and head that should roll should be the one that published the chatbot.
But it's important to acknowledge that there was a 'bug' in an underlying tool and not in the chatbot, and still PIP/fire those responsible for publishing the chatbot and exposed an otherwise internal tool to the public, and not those that introduced the 'bug' to an internal tool.
That sounds a lot like the justifications Claude and ChatGPT give when confronted about something they did wrong, or when asked to provide a customer support response about software issues
I've lost track of the number of times Claude has basically said "it was like that when i got here" in the face of a clearly bogus choice and easily disproved explanation.
The argument here is that the AI is a glorified input page. The input field asks for your username and email and sends it to a backend function. Such an input page is working as intended.
The problem is when the backend function doesn't verify that the email matches the username.
Why on earth would the backend function even take an email?
Or perhaps said different: use the submitted info to identify the account; send any sensitive messages (recovery codes, password resets whatever) to only the contact info on file. If the chat bot can send such email it should do so via an API that sends only to contact info on file for the associated account and not to an email that's provided by the bot.
> Why on earth would the backend function even take an email?
In principle, it could be designed to do so to handle cases where a new email address has been confirmed out of band, e.g. for an account representing an organization or political office. But that's a relatively unusual situation, not something you'd want to be available to every user writing in. (Even if you had an all-human support department, this sort of functionality would only be available to a select few agents.)
I like to dunk on Meta as much as the next guy, but I think this makes sense: deterministic verification like this is not, and should never be, the LLM’s job. The tools it has access to should enforce the permissions layer, ensuring that the LLM can never perform actions the user themselves should not be allowed to perform. In this case, the tool failed to do that.
so how long was the bug there? was there a way to access it before/without the support agent? it feels like Meta will throw anything under the bus to redirect blame from the AI, because that would be the end of their $600B (depending on “which number you want to go with”) experiment
What I gather is that this internal tool was used by human support agents, and it was their responsibility to verify the email adresses and general validity of a claim.
But when implementing AGI TM that was overseen, maybe the oversight in the separate code path was a 'bug', but the mistake was making the chatbot obviously, if the separate code path had a bug, then it had become ossified into a feature, and it was internal, not exposed to the public.
This is an external communication, to save face sure, but if this is the internal excuse, it would be absolutely the wrong RCA and it reads as if the one who made the mistake is not admitting they made their mistake. Which to be honest, just making the mistake is enough to get fired, but not admitting it is enough to get ultra fired.
Did you create the account separately? Or as an asset of your main Meta account (like Meta Business Suite)?
I'm creating the accounts in Meta Business Suite, so I would have a recourse with my main personal account which can be linked to some adspend, so I'm assuming it will have better support channels than accounts created through an end-user interface.
Why was 'can a user request a different email' not literally the first test that comes to mind when making something like this? Do they not test anything because the scale is too big?
If this was a bank that had zero humans and the AI chatbot was abused to hand over sensitive information about their customers which led to this disaster, people would never trust their bank ever again and leave.
Meta believes that they can vibe-code their reputation down the drain by removing humans in the loop.
Applying a technical solution to a social problem almost always ends in disasters like this.
> "The tool itself worked properly and functioned as intended; however due to a bug in a separate code path, the system did not properly verify that the email address provided by the individual requesting a password reset matched the email address associated with that user’s Instagram account," said Meta in its breach notice.
I'm not sure "worked properly" and "as intended" accurately describe this situation.
In italian we say "l'operazione è riuscita perfettamente, ma il paziente è morto" -> "the surgery was a complete success, but the patient died"
Both this and what Meta said reminds me of "Clarke and Dawe - The Front Fell Off" (https://www.youtube.com/watch?v=3m5qxZm_JqM)
I also can't believe the people who were involved with writing this response from Meta, didn't realize how obviously bad it sounds. It's like there is no humans working and writing there anymore.
The tool worked correctly and as intended, but due to a bug it did not work correctly nor as intended.
To be fair, that quote in the original article could have more context. By "The tool" they meant "AI-assisted support tool"[1]; perhaps they meant that the issue was not an AI hallucination inherent of the tool, but a fixable bug.
[1]: https://www.documentcloud.org/documents/28202858-meta-ai-ag-...
In that case, the statement is so meaningless as to be useless. Why should we care how Meta splits up their microservices? The tool still failed. They just want to redefine the "tool" as something else, anything else, to avoid having to admit something negative about our precious AI.
> The LLM correctly generated tokens according to user input, however due to a bug in a separate code path, the system did not properly verify the email address
> Nginx correctly handled the user requests according to the HTTP standard, however due to a bug in a separate code path, the system did not properly verify the email address
Sounds like they are saying the agent did not malfunction, and this vuln could have been triggered by a human support agent too.
It probably could have been, but how likely is that compared to with the AI agent? I'd assume (and I'm ready to look like an idiot if I'm wrong) that the humans are trained to send the verification code to the email address on file, rather than any address the client asks them to. I'd certainly assume most of them are more afraid of the consequences than the AI is.
I get the joke, but it's a relevant nuance that the new code, the chatbot, did not have 'the bug'. I still think that the mistake and head that should roll should be the one that published the chatbot.
But it's important to acknowledge that there was a 'bug' in an underlying tool and not in the chatbot, and still PIP/fire those responsible for publishing the chatbot and exposed an otherwise internal tool to the public, and not those that introduced the 'bug' to an internal tool.
That sounds a lot like the justifications Claude and ChatGPT give when confronted about something they did wrong, or when asked to provide a customer support response about software issues
I've lost track of the number of times Claude has basically said "it was like that when i got here" in the face of a clearly bogus choice and easily disproved explanation.
Oh it was a downstream dependency. The tool worked, it was the downstream dependency. Glory to Arstotszka
The argument here is that the AI is a glorified input page. The input field asks for your username and email and sends it to a backend function. Such an input page is working as intended.
The problem is when the backend function doesn't verify that the email matches the username.
Why on earth would the backend function even take an email?
Or perhaps said different: use the submitted info to identify the account; send any sensitive messages (recovery codes, password resets whatever) to only the contact info on file. If the chat bot can send such email it should do so via an API that sends only to contact info on file for the associated account and not to an email that's provided by the bot.
> Why on earth would the backend function even take an email?
In principle, it could be designed to do so to handle cases where a new email address has been confirmed out of band, e.g. for an account representing an organization or political office. But that's a relatively unusual situation, not something you'd want to be available to every user writing in. (Even if you had an all-human support department, this sort of functionality would only be available to a select few agents.)
Fair enough. Never trust client-submitted browser form, but always trust LLM-submitted form.
I like to dunk on Meta as much as the next guy, but I think this makes sense: deterministic verification like this is not, and should never be, the LLM’s job. The tools it has access to should enforce the permissions layer, ensuring that the LLM can never perform actions the user themselves should not be allowed to perform. In this case, the tool failed to do that.
>deterministic verification like this is not, and should never be, the LLM’s job.
But when humans handled it, this was not as much as a problem. That is, the humans did the job, because they recognized the need to do that job.
Sure sometimes accounts could get recovered if a human was tricked, but evidently it was easier to trick the LLM in masse than humans.
Maybe they’re communicating exactly what it sounds like and are just owning up to being complete morons?
How very Wernher von Braun of them.
so how long was the bug there? was there a way to access it before/without the support agent? it feels like Meta will throw anything under the bus to redirect blame from the AI, because that would be the end of their $600B (depending on “which number you want to go with”) experiment
Then ‘ The tool itself’ was not appropriate to the job in the first place
Of course.
What I gather is that this internal tool was used by human support agents, and it was their responsibility to verify the email adresses and general validity of a claim.
But when implementing AGI TM that was overseen, maybe the oversight in the separate code path was a 'bug', but the mistake was making the chatbot obviously, if the separate code path had a bug, then it had become ossified into a feature, and it was internal, not exposed to the public.
This is an external communication, to save face sure, but if this is the internal excuse, it would be absolutely the wrong RCA and it reads as if the one who made the mistake is not admitting they made their mistake. Which to be honest, just making the mistake is enough to get fired, but not admitting it is enough to get ultra fired.
Meanwhile an account I created for a new product was permanently disabled by an automated system with no path for me to appeal to a human.
(If anyone at Meta/Instagram sees this I wrote a brief blog post with the details. Please help! https://addisonwebb.com/blog/2026-06-05-Can%20Someone%20at%2... )
Did you create the account separately? Or as an asset of your main Meta account (like Meta Business Suite)?
I'm creating the accounts in Meta Business Suite, so I would have a recourse with my main personal account which can be linked to some adspend, so I'm assuming it will have better support channels than accounts created through an end-user interface.
This was on hacker news a few days ago (https://news.ycombinator.com/item?id=48359102) - description of the “hack”, not the cockamamie confirmation by Meta.
Why was 'can a user request a different email' not literally the first test that comes to mind when making something like this? Do they not test anything because the scale is too big?
In their defense, they asked the LLM to make no mistakes
>AI-assisted account recovery system
oh no...Meta what are you doing
Corrected headline: "Meta confirms 1000s of Instagram accounts were hacked due to their insecure AI chatbot".
Probably some product manager pushed back on security considerations raised by engineers.
"abusing" by using it's built in insecurity to do insecure things.
It's like, people abusing an open door. "Guys, just because we left the door open to your bedroom doesn't mean we're responsible".
God can only hope this is a business ending lawsuit.
If this was a bank that had zero humans and the AI chatbot was abused to hand over sensitive information about their customers which led to this disaster, people would never trust their bank ever again and leave.
Meta believes that they can vibe-code their reputation down the drain by removing humans in the loop.
Applying a technical solution to a social problem almost always ends in disasters like this.
Reputation can’t be vibe-coded.
Meta's brand is already toxic. Idk if there's much to lose there.
https://www.documentcloud.org/documents/28202858-meta-ai-ag-...
https://www.maine.gov/agviewer/content/ag/985235c7-cb95-4be2...
> Date(s) Breach Occured: 04/17/2026
> Date Breach Discovered: 05-31-2026