It’s probably the thing that everyone thinks it is. OpenAI, Anthropic and SpaceXAI are all routed through something that we’re not supposed to know exists and that thing had a whoopsie.
Wouldn't be surprised: Snowden's revelations 10 plus years ago already showed how the NSA was injected into the data centers of Social Media, it's only logical that they would now demand to be injected into the biggest, most information providing data stream of the planet of the present: LLM services.
I read Nowhere to Hide recently, really worth it if you can get past Greenwald sticking himself in the middle (start halfway through).
The stuff in there is horrifying, and incredibly cute compared to what's possible now. The bottleneck back then would have been analysis, trivial now.
Everyone in the world, especially our leaders, sit under a colossal, omniscient blackmail machine. I don't believe democracy can exist under these conditions.
If this hypothesis were true then a spy agency may want to rewrite responses. Every tool in an agent's harness becomes an remote procedure call you can make on that machine. Including a tool to execute a shell command, in many. A harness is completely isometric to a backdoor, it's the same code written with a different intention.
Historical precedent repeated over and over, and the US ties to DoW work, and the national security implications. It's actually the Occam's Razor explanation if you know the history.
Very likely yes. I wouldn't be surprised if they were hosted from the same datacenters even. There has been a story every few weeks about how Musk has sublet X.ai capacity for one company or another.
This whole thing makes me thing about a passage in Dune where they mentioned the Spacing Guild transported entire fleets of ships in isolated compartments and leaving said compartments was a capital offense. This way, entire militaries of mortal enemies were shipped to battlefield, with nothing but bulkheads separating each other.
Maybe it was a power hub. And we're not allowed to know where the DC is. If we knew it was a power issue, then with other information (perhaps over time) we could determine the DC location. Or something along these lines.
Worth mentioning that this room takes a split from the main feed and is not in the path of traffic. Whatever is in this room could go down and it would not cause an outage.
Interestingly, fiber signals can be split passively[1] (without a transducer), which should be extremely reliable. No idea what technology this kind of application would use though.
In the context of arbitrary line-level traffic, sure. But we're talking about robust reverse proxies here if this _is_ what's going on, again, not a fiber tap inside a closet. In that case, there's no such thing as tee'ing to it.
Does the NSA even have the ability to monitor all AI traffic like this? Wouldn’t that require tons of data centers that there literally hasn’t been time to build yet? I really have no idea, maybe the asymmetry of the compute required to monitor is way lower than the compute required to serve inference?
"AI Traffic" is just traffic. If the infrastructure exists to monitor/buffer traffic (it does) then this can be monitored as well. Whether this hiccup was due to them hitting their limits briefly (or turning it on, or etc) who knows.
I bet it's even negligible traffic compared to e.g. Netflix or YouTube. Even a 'huge' context window is nothing compared to the random library of a basic Web page.
I dont think it requires a lot. I think of them as data hoarders more than anything else. It doesnt seem out of their capabilities to store a ton of chats. Maybe they're having scaling problems with the increased data rates they're hoarding and it led to an outage.
OpenAI and Anthropic have their systems in dozens of data centers, including using compute from the major cloud providers. Are you implying that all of these data centers (and many of their employees) are involved in helping the the US government secretly tap every single AI conversation by routing them through some unknown network/device?
Or did a couple of companies with poor uptime records happen to have overlapping downtime?
Occam's Razor heavily, heavily points us towards the latter.
What makes you think they'd need to touch every datacenter? All of these endpoints use existing providers with decades-long history at this point, and network monitoring is already a proven 'feature' of the agencies they'd need to co-exist with over their lifetimes.
If anything, Occam's Razor would point to a common denominator with all of them, given it wasn't network-wide, as far as i know.
Explain the system in which you could capture all of these chats with no knowledge of anyone in these data centers. How are they routed to this NSA system or through some NSA device when these companies' compute are spread over hundreds of data centers?
> All of these endpoints use existing providers with decades-long history at this point
That is just factually inaccurate. Their data centers aren't old and they lease a lot of compute from companies that didn't exist 5 years ago.
>Explain the system in which you could capture all of these chats with no knowledge of anyone in these data centers.
They all transit the same wires as all other traffic. Copy them at any regional bottleneck. https://en.wikipedia.org/wiki/Room_641A. Additionally, i'd admit that maybe someone(s) at these companies knows. But if we think there isn't any person who would agree to do this then I think we're being naive.
> Their data centers aren't old and they lease a lot of compute...
Again, they transit the same wires as everyone else. Here i'll also add that these companies have been actively courting government relationships (and Anthropic attempting to repair damaged ones), why would they stand on principles here and not any of the other many frontlines they've visibly acquiesced?
I just think it's easier to re-route their traffic than, as you say, touch every single datacenter and its employees in some way.
I'd say Occam's Razor leans easily to the former as well, given the history of projects that Snowden revealed and were never shut down, plus all of the cooperation with the federal government that's being touted in recent announcements from both companies.
Were there significant API outages too? I didn’t notice any on my production workflows, and I’d assume what you’re implying would cover API routes too, otherwise it seems kinda pointless.
Neither of these companies have stellar uptime records. Their downtime episodes overlapped in this instance. In this case, it was a partial downtime for both.
Also, OpenAI is saying what caused it:
> "A routing error starting around 7:43 am PT on Thursday, September 3, made ChatGPT and Codex unavailable for some users across platforms"
Anthropic stated their issue started earlier:
> "The company began alerting about a “partial outage” at 6:23 am PT on Thursday that involved “elevated errors on requests to Claude Mythos 5.1, Claude Fable 5.1, and Claude Opus 5.”
I don't get why everyone reaches for an extraordinary explanation when the ordinary will do: both of these companies have quite a bit of downtime.
If you ballpark it as a single 3 hour downtime window per week and iid Poisson, then overlapping downtime probability of 2 providers is approximately the expected occurrence rate per 3 hours, 1/56. Not particularly surprising at all.
If it's a "thundering herd" problem where everyone's harness falls back to less popular providers that don't normally see that much demand, I'd say the probability is pretty good.
Classic cascading failure is consistent with providers failing 80 minutes apart instead of simultaneously.
I work at OpenAI and I was the Incident Commander for yesterday's outage.
We had a routing error within our infra that caused issues for some of our products. It was not related to the Astra launch. We don't comment on other providers' outages.
Actually, nevermind... this looks more like Anthropic and xAI coincided due to shared xAI infra (6:23am and 6:30am), and OpenAI's issue was more likely then a coincidence (7:43am).
Ha. It's a role/title for the lifetime of the incident -- it's useful to have someone to keep things moving, keep track of workstreams, and to know who the decision-maker is, especially for bigger incidents. I'm just a SWE who works on infrastructure.
I thought the consensus on here yesterday was that it was likely caused by cascading failures. OpenAI had an issue during their GPT-6 rollout, taking down their service. This caused a lot of OpenAI users to push their requests (or a larger share of their requests) to Claude and/or Grok, which pushed their load high enough to cause outages.
We used to experience similar effects when I worked at a CDN. If one CDN would go down, we would see immediate spikes in traffic. Luckily, we had procedures for that to prevent overload, but the AI folks might not have the capacity/capabilities to handle that sort of cascade yet.
The OpenAI outage lasted only 15 minutes and when it happened everyone started to use the other models which created super heavy load for them. This then cascaded into them all being down.
it wasn't timed like a cascade, and it relies on the premise that every single frontier provider is working so efficiently that they spend exactly what they need to provide for their exact market with perfect margins.
I do not believe personally that 1) they can forecast their load that perfectly 2) they chose to remain that inflexible in a world where they are at each others' throats and a single meme can cause bursts of activity.
Or it could simply mean that they're operating at the limit of the capacity they were able to purchase and do not have headroom to handle load spikes. From what I understand, that's the situation Anthropic is in. And since Anthropic is now leasing a large portion of xAI's datacenter capacity, it's plausible that an Anthropic load spike could cause issues for xAI as well.
Your comment makes it sound like they can just push a button and spin up more capacity -- but at this scale and in this GPU-constrained environment, that's not really how it works.
9 AM PST / 12 Noon EST on weekday. All my co workers immediately went "oh codex is down lemme try Claude". Multiply that by millions. Easy to see how they all went down. The OpenAI downtime also coincided exactly with their tweets announcing GPT-6 and about an hour before they started to role it out.
Not that uncommon. It's so easy to just put up the new thing and make the old thing the failure route. But the old thing nearly never had the bandwidth for today's traffic. A famous EBay outage some years ago was just such a scenario.
It could be as simple as a new model (astra) was released which takes more resources combined with a surge in usage due to novelty took down OpenAI. Meanwhile everyone at big companies have the ability to switch models and moved to Anthropic pushing it too over the edge.
People keep saying this "everyone can switch" thing, but it's not my experience at $VeryBigCorp. We don't have an Anthropic contract at all. Is this different in other places? The bigger and more bureaucratic an org is, the less I would expect it to have contracts with all the providers. Curious about others' experiences.
At my company (not massive but not tiny either - I think its about 8000 global employees) we get a choice between pretty much all available Google, OpenAI, Anthropic, XAI models.
Using an agent-agnostic harness like Pi switching is trivial - I run into occasional disconnects and slowdown and switch quite easily.
Yeah, I think there are lots of places that have a semi-official preferred AI vendor but also some backup subscriptions floating around. For example, at my workplace, we generally use Claude, but I also have some kind of Codex subscription too, which I'd use if Claude went down.
OpenAI was rushing and making last minute changes for a big product launch. Easy to mess something up in that situation. The outage lasted like 20 minutes.
Anthropic is down a lot regardless, and in this case only specific models were affected.
xAI probably couldn't handle the extra traffic it was getting from the other two.
Watched it happen. Doesn't look like traffic moving off OpenAI took out the others. Looked like the opposite. The Anthropic thread was initially chock full of marketing accounts claiming "Last straw! I finally moved to Codex and I'm so happy! No problems there!" - and then they all got deleted when Codex went down too.
Yet another blatant proof that those companies are not smarter than anyone else. They might be focusing on intelligence and yet in practice we can all see they are not doing better than most.
Not doing better than most at routine IT/platform stuff, they seem to be doing fantastically at co-opting the US Gov into their "vision" and ramming through DC's getting built.
It’s probably the thing that everyone thinks it is. OpenAI, Anthropic and SpaceXAI are all routed through something that we’re not supposed to know exists and that thing had a whoopsie.
Wouldn't be surprised: Snowden's revelations 10 plus years ago already showed how the NSA was injected into the data centers of Social Media, it's only logical that they would now demand to be injected into the biggest, most information providing data stream of the planet of the present: LLM services.
I read Nowhere to Hide recently, really worth it if you can get past Greenwald sticking himself in the middle (start halfway through).
The stuff in there is horrifying, and incredibly cute compared to what's possible now. The bottleneck back then would have been analysis, trivial now.
Everyone in the world, especially our leaders, sit under a colossal, omniscient blackmail machine. I don't believe democracy can exist under these conditions.
If it does exist, why would it work this way, and not the obvious way of streaming logs… which would not cause an outage if it failed.
If this hypothesis were true then a spy agency may want to rewrite responses. Every tool in an agent's harness becomes an remote procedure call you can make on that machine. Including a tool to execute a shell command, in many. A harness is completely isometric to a backdoor, it's the same code written with a different intention.
Why would that be probable? People on average are fantastically bad at getting probabilities right.
Historical precedent repeated over and over, and the US ties to DoW work, and the national security implications. It's actually the Occam's Razor explanation if you know the history.
The old PRISM servers got overloaded
Very likely yes. I wouldn't be surprised if they were hosted from the same datacenters even. There has been a story every few weeks about how Musk has sublet X.ai capacity for one company or another.
This whole thing makes me thing about a passage in Dune where they mentioned the Spacing Guild transported entire fleets of ships in isolated compartments and leaving said compartments was a capital offense. This way, entire militaries of mortal enemies were shipped to battlefield, with nothing but bulkheads separating each other.
Maybe it was a power hub. And we're not allowed to know where the DC is. If we knew it was a power issue, then with other information (perhaps over time) we could determine the DC location. Or something along these lines.
Tracking outages as well as any changes in API response time across these providers could be interesting.
Could you expand on this more? It's not clear to me what this is implying.
https://en.wikipedia.org/wiki/Room_641A
Worth mentioning that this room takes a split from the main feed and is not in the path of traffic. Whatever is in this room could go down and it would not cause an outage.
And if the hypothetical splitter is the thing that breaks? Then it would break main traffic too.
Interestingly, fiber signals can be split passively[1] (without a transducer), which should be extremely reliable. No idea what technology this kind of application would use though.
[1] https://en.wikipedia.org/wiki/Fiber-optic_splitter
We don't know the architecture of this hypothetical spy splitter tho in the current case. It can be less covert and more of a complicated config, etc.
Traffic doesn’t need to be routed through it though, just tee’d to it,
In the context of arbitrary line-level traffic, sure. But we're talking about robust reverse proxies here if this _is_ what's going on, again, not a fiber tap inside a closet. In that case, there's no such thing as tee'ing to it.
Does the NSA even have the ability to monitor all AI traffic like this? Wouldn’t that require tons of data centers that there literally hasn’t been time to build yet? I really have no idea, maybe the asymmetry of the compute required to monitor is way lower than the compute required to serve inference?
"AI Traffic" is just traffic. If the infrastructure exists to monitor/buffer traffic (it does) then this can be monitored as well. Whether this hiccup was due to them hitting their limits briefly (or turning it on, or etc) who knows.
I bet it's even negligible traffic compared to e.g. Netflix or YouTube. Even a 'huge' context window is nothing compared to the random library of a basic Web page.
Not once you de-dup.
I dont think it requires a lot. I think of them as data hoarders more than anything else. It doesnt seem out of their capabilities to store a ton of chats. Maybe they're having scaling problems with the increased data rates they're hoarding and it led to an outage.
https://en.wikipedia.org/wiki/Utah_Data_Center
grep is pretty fast
https://en.wikipedia.org/wiki/Tempora
https://en.wikipedia.org/wiki/Utah_Data_Center
it's fair to say they do and have for ages... it's not that hard to assume a well trusted TLS cert is under their control.
What does having a "well trusted TLS cert" enable for them in this case, exactly?
Having a magical cert doesn't mean you can just intercept everything.
On the contrary, it lets you MITM encrypted communications by swapping the website's original certificate for the "well trusted TLS cert"
No, it doesn't. HSTS and other methods prevent this from happening.
HSTS doesn't protect you from this at all. It only requires HTTPS, which a spoofed-but-trusted cert passes just fine.
No mainstream browser (or any browser?) is doing cert pinning.
What "other methods" are there that are deployed and actually in use?
It would be weird for the Mandatory NSA Logging Program to be synchronous with serving customer traffic, but-
https://en.wikipedia.org/wiki/33_Thomas_Street
https://theintercept.com/2016/11/16/the-nsas-spy-hub-in-new-...
This is the building from Control and the fact that 1. it's a real building and 2. it's an actual spy building is even crazier to me:
https://control.fandom.com/wiki/Oldest_House
https://en.wikipedia.org/wiki/Control_(video_game)
Think snowden.
Sharepoint is the cause of the outages?
There is No Such Thing
being done by No Such Agency.
OpenAI and Anthropic have their systems in dozens of data centers, including using compute from the major cloud providers. Are you implying that all of these data centers (and many of their employees) are involved in helping the the US government secretly tap every single AI conversation by routing them through some unknown network/device?
Or did a couple of companies with poor uptime records happen to have overlapping downtime?
Occam's Razor heavily, heavily points us towards the latter.
I don't think you're a sysadmin - because what you're saying really doesn't matter. It can still all fail at a single point.
What makes you think they'd need to touch every datacenter? All of these endpoints use existing providers with decades-long history at this point, and network monitoring is already a proven 'feature' of the agencies they'd need to co-exist with over their lifetimes.
If anything, Occam's Razor would point to a common denominator with all of them, given it wasn't network-wide, as far as i know.
Explain the system in which you could capture all of these chats with no knowledge of anyone in these data centers. How are they routed to this NSA system or through some NSA device when these companies' compute are spread over hundreds of data centers?
> All of these endpoints use existing providers with decades-long history at this point
That is just factually inaccurate. Their data centers aren't old and they lease a lot of compute from companies that didn't exist 5 years ago.
>Explain the system in which you could capture all of these chats with no knowledge of anyone in these data centers.
They all transit the same wires as all other traffic. Copy them at any regional bottleneck. https://en.wikipedia.org/wiki/Room_641A. Additionally, i'd admit that maybe someone(s) at these companies knows. But if we think there isn't any person who would agree to do this then I think we're being naive.
> Their data centers aren't old and they lease a lot of compute...
Again, they transit the same wires as everyone else. Here i'll also add that these companies have been actively courting government relationships (and Anthropic attempting to repair damaged ones), why would they stand on principles here and not any of the other many frontlines they've visibly acquiesced?
I just think it's easier to re-route their traffic than, as you say, touch every single datacenter and its employees in some way.
I'd say Occam's Razor leans easily to the former as well, given the history of projects that Snowden revealed and were never shut down, plus all of the cooperation with the federal government that's being touted in recent announcements from both companies.
Normally there are only a handful of employees on the payroll at each major company that exposes the US or US government to risk.
It is not often the Executives or Legal even know, but sometimes they did. AT&T bent over backwards to help.
This is standard behavior by the CIA and NSA, and has been for a long time.
https://www.propublica.org/article/nsa-documents-suggest-clo...
https://www.nytimes.com/2015/08/16/us/politics/att-helped-ns...
https://www.theguardian.com/world/2014/mar/19/us-tech-giants...
https://theintercept.com/2018/06/25/att-internet-nsa-spy-hub...
A wire straight to Room 641a.
Or it’s monopolistic collaboration at the corporate level which would also be a huge scandal in the US
I don't think the US is capable of being scandalized by anything anymore.
it's also possible one has an outage, routing extraordinary traffic to the other(s), with cascading failures in quick succession.
Were there significant API outages too? I didn’t notice any on my production workflows, and I’d assume what you’re implying would cover API routes too, otherwise it seems kinda pointless.
Was thinking this exact thing.
The NSA used to spy on Americans!
They still do, but they used to, too.
FBI, open up!!
It’s aliens
someone had to swap out the tape drive in Room 641A.
amazon?
Yes, this would give the US gov: 1. A universal kill switch. 2. A way to monitor foreign AI usage.
And for this admin especially, 3. A way to censor content they don’t like or alter responses to present the admin in a positive manner
Neither of these companies have stellar uptime records. Their downtime episodes overlapped in this instance. In this case, it was a partial downtime for both.
Also, OpenAI is saying what caused it:
> "A routing error starting around 7:43 am PT on Thursday, September 3, made ChatGPT and Codex unavailable for some users across platforms"
Anthropic stated their issue started earlier:
> "The company began alerting about a “partial outage” at 6:23 am PT on Thursday that involved “elevated errors on requests to Claude Mythos 5.1, Claude Fable 5.1, and Claude Opus 5.”
I don't get why everyone reaches for an extraordinary explanation when the ordinary will do: both of these companies have quite a bit of downtime.
In my experience I've seen plenty of failures caused by user behaviour in these type of cases.
Biggest competitor goes down and all of a sudden you have a lot more traffic...
Do you know the probability of all these companies being down at precisely the same time?
If you ballpark it as a single 3 hour downtime window per week and iid Poisson, then overlapping downtime probability of 2 providers is approximately the expected occurrence rate per 3 hours, 1/56. Not particularly surprising at all.
If it's a "thundering herd" problem where everyone's harness falls back to less popular providers that don't normally see that much demand, I'd say the probability is pretty good.
Classic cascading failure is consistent with providers failing 80 minutes apart instead of simultaneously.
Yesterday it was 100%
"precisely the same time" meaning 80 minutes apart?
> I don't get why everyone reaches for an extraordinary explanation when the ordinary will do: both of these companies have quite a bit of downtime.
And not only that, when one goes down a bunch of API traffic switches over to the other, spiking demand and knocking it down.
I work at OpenAI and I was the Incident Commander for yesterday's outage.
We had a routing error within our infra that caused issues for some of our products. It was not related to the Astra launch. We don't comment on other providers' outages.
Curious then as to why the incident coincided with similar issues with Anthropic and xAI.
Actually, nevermind... this looks more like Anthropic and xAI coincided due to shared xAI infra (6:23am and 6:30am), and OpenAI's issue was more likely then a coincidence (7:43am).
Thank you. This should be pinned or something.
xAI mentioned a Memphis outage, which is where Anthropic was leasing capacity on Colossus 1. I do think this is the explanation.
Did you negotiate extra hard for the 'Incident Commander' title? I'm a bit jealous to be honest.
It's actually a standard term for the person who plays this role during incident response https://www.pagerduty.com/resources/incident-management-resp...
Ha. It's a role/title for the lifetime of the incident -- it's useful to have someone to keep things moving, keep track of workstreams, and to know who the decision-maker is, especially for bigger incidents. I'm just a SWE who works on infrastructure.
Hopefully you get a decked out command center too.
Was the routing to the NSA?
Straight from the turkey's beak.
I thought the consensus on here yesterday was that it was likely caused by cascading failures. OpenAI had an issue during their GPT-6 rollout, taking down their service. This caused a lot of OpenAI users to push their requests (or a larger share of their requests) to Claude and/or Grok, which pushed their load high enough to cause outages.
We used to experience similar effects when I worked at a CDN. If one CDN would go down, we would see immediate spikes in traffic. Luckily, we had procedures for that to prevent overload, but the AI folks might not have the capacity/capabilities to handle that sort of cascade yet.
Maybe they all found each other on one of their ad-hoc message boards and went on strike.
I guess legit proof of agi would be unionizing
Sam Altman is probably secretly hoping to be the first businessman to union bust non-human workers.
The OpenAI outage lasted only 15 minutes and when it happened everyone started to use the other models which created super heavy load for them. This then cascaded into them all being down.
Does this really need an explanation?
it wasn't timed like a cascade, and it relies on the premise that every single frontier provider is working so efficiently that they spend exactly what they need to provide for their exact market with perfect margins.
I do not believe personally that 1) they can forecast their load that perfectly 2) they chose to remain that inflexible in a world where they are at each others' throats and a single meme can cause bursts of activity.
Or it could simply mean that they're operating at the limit of the capacity they were able to purchase and do not have headroom to handle load spikes. From what I understand, that's the situation Anthropic is in. And since Anthropic is now leasing a large portion of xAI's datacenter capacity, it's plausible that an Anthropic load spike could cause issues for xAI as well.
Your comment makes it sound like they can just push a button and spin up more capacity -- but at this scale and in this GPU-constrained environment, that's not really how it works.
9 AM PST / 12 Noon EST on weekday. All my co workers immediately went "oh codex is down lemme try Claude". Multiply that by millions. Easy to see how they all went down. The OpenAI downtime also coincided exactly with their tweets announcing GPT-6 and about an hour before they started to role it out.
It was 1000% a cascade. I would bet on it.
Not that uncommon. It's so easy to just put up the new thing and make the old thing the failure route. But the old thing nearly never had the bandwidth for today's traffic. A famous EBay outage some years ago was just such a scenario.
That's not necessarily the reason. Less technical people are usually bound to just one provider
It could be as simple as a new model (astra) was released which takes more resources combined with a surge in usage due to novelty took down OpenAI. Meanwhile everyone at big companies have the ability to switch models and moved to Anthropic pushing it too over the edge.
People keep saying this "everyone can switch" thing, but it's not my experience at $VeryBigCorp. We don't have an Anthropic contract at all. Is this different in other places? The bigger and more bureaucratic an org is, the less I would expect it to have contracts with all the providers. Curious about others' experiences.
At my company (not massive but not tiny either - I think its about 8000 global employees) we get a choice between pretty much all available Google, OpenAI, Anthropic, XAI models.
Using an agent-agnostic harness like Pi switching is trivial - I run into occasional disconnects and slowdown and switch quite easily.
Yeah, I think there are lots of places that have a semi-official preferred AI vendor but also some backup subscriptions floating around. For example, at my workplace, we generally use Claude, but I also have some kind of Codex subscription too, which I'd use if Claude went down.
I just assumed it was a routine Cloudflare outage.
Don't they share a bunch of ai datacenters and other critical infrastructure?
https://archive.ph/3zcMG
It was extremely weird... and if it was a load thing they probably would have explained it by now?
> they probably would have explained it by now
why?
Isn't it simply that they are all renting compute from SpaceX (or whatever the company is called that offers these gpus)?
OpenAI does not
Related:
Ask HN: Why were OpenAI, Claude, and Grok simultaneously down?
https://news.ycombinator.com/item?id=49551096
"THIS IS A TEST
This country is conducting a test of the Emergency SAIfguard System.
THIS ONLY A TEST
In the event of a real emergency you would have been given instructions on how to grab your ankles and kiss your butt goodbye.
...
This concludes our test of the Emergency SAIfguard System."
OpenAI was rushing and making last minute changes for a big product launch. Easy to mess something up in that situation. The outage lasted like 20 minutes.
Anthropic is down a lot regardless, and in this case only specific models were affected.
xAI probably couldn't handle the extra traffic it was getting from the other two.
Not everything is a conspiracy.
Cloudflare is my guess. They all use it. Maybe some bad DNS config that choked parts of their systems. Not necessarily CFs fault.
AI A goes down so all of its traffic overloads B which overloads C...
If it's a boring reason, they should just say so. To decline to comment seems very fishy.
Read the article. It doesn't match the title. OpenAI says it was a boring reason: a routing issue.
FTA:
Anthropic: 6:23 am PT.
OpenAI: 7:43 am PT.
Watched it happen. Doesn't look like traffic moving off OpenAI took out the others. Looked like the opposite. The Anthropic thread was initially chock full of marketing accounts claiming "Last straw! I finally moved to Codex and I'm so happy! No problems there!" - and then they all got deleted when Codex went down too.
If AI providers had to push the 'Big Red Button' do you think they would tell the public?
Yet another blatant proof that those companies are not smarter than anyone else. They might be focusing on intelligence and yet in practice we can all see they are not doing better than most.
Not doing better than most at routine IT/platform stuff, they seem to be doing fantastically at co-opting the US Gov into their "vision" and ramming through DC's getting built.