mythos allowed mediocre people to get results by holding their hand through the process, or just ignoring their irrelevant input and knowing what to do.
if you throw millions of tokens at IDA Pro MCP with the right prompt lets just say security by obscurity fails miserably because there is no obscurity when the LLM chews through the decompilation.
The NSA is managed by the NSA director, an independently appointed and confirmed office separate from the Office of the Director of National Intelligence. The DNI does have the authority to restrict funds to the NSA, and sets certain priorities, policy, etc. but the NSA director is not beholden to the DNI and makes their hiring and firing decisions independently. They’re also, currently and historically though not required by statute, a flag officer in the US military and dual-hatted as the commander of CYBERCOM. All this is to say, chaos in the office of the DNI does not necessarily impact the NSA.
You're assuming that they are "dismantling" it in a sense, what they are actually doing is mostly attacking workers while introducing extremely unsafe software. If you think LLMs are terrible, imagine it being the gatekeeper on whether your personal info is shared to an individual (and they ain't wasting time on the boring info either!).
When you want to reorient the government, it's much easier doing it with a smaller more loyal force. Now introduce tools that make mass surveillance easier and less accountable.
Like that's not a bad thing for them, that's what they want to do.
---
Back to the article, I'm not shocked that a massive LLM company speed running into the brick wall that is the US government; just thought it would be OpenAI, but Sam Altman is truly the best bottom feeder the game.
Also fully believe that Anthropic is hoping that public sentiment is on their side but more Americans hate AI companies than Trump so it's not going to go how they want.
Give it maybe 3-6 months before the Trump Admin talks about openly nationalizing Anthropic.
Anecdata suggests NSA just got on board and kept going tbh. Not sure they’ve felt the same impacts / churn as other agencies, and not sure they’ve ever really been that beholden to the DNI.
Seems to me OP's implication is that they were fired because someone wanted to hit a quota of (employees cut/payroll expenses reduced), or other similarly ''reasonless'' justifications.
AI marketing bullshit stunts are unlike anything I've seen in 30 years. It started with MS Copilot so called capabilities for work, which were completely made up use cases that didn't work at all (3 years later still). We've had OpenAI "AGI is coming" and "AI will take your job", now we have Mythos being so "dangerous" for cybersecurity, which of course makes the average Joe interpret it as Anthropic being "the better overall company, the NSA uses it!!". I mean gov foes with Anthropic are probably true, but the marketing is to blame not Mythos capabilities. This is all so fucking pathetic
Don't forget, its no longer cool to say that now that the public has pushed back. The fact they all changed their tone away from taking jobs tells you that it was all just entirely marketing.
But the propaganda deluge was a smash hit so far, HN is drowning in “AI” BS, and astroturfers and spin doctors haven’t seen that much business since the cold war. They made more profit than shovel salesmen in the gold rush.
We should seriously reframe this whole AI thing to "SI = simulated intelligence".
It's google in a box. Great achievement, makes knowledge work faster, but please stop bothering everyone else.
The Uber and Groupon people became billionaires, so the "Simulated Intelligence" folks will also achieve it. No need to worry and drown everyone in these bs stories only non-tech people believe.
Heh. In the Schlock Mercenary universe, "SI" means "synthetic intelligence", which is a level below real AI (which means what we would call AGI). And, as it says (in https://www.schlockmercenary.com/2003-07-21), SI translates to "kinda stupid".
I was able to identify, diagnose, fix, and upstream a minor bug in and erlang/OTP ssh key implementation with Opus in maybe 20 minutes (+2 weeks or so for upstream). It is not impossible that I could have done this before, but it would have taken days or weeks. The actual fix was about 2 lines of code, hardly AI slop, but getting there would have been quite the slog, and I never would have done it.
There is a lot of the reason for AI skepticism out there, but people tend to do massive overcorrections and underestimate the force multiplier it can be, particularly for people with some idea of what they're doing and a good grasp of how to take advantage of the tool.
So let’s say you’re in Anthropic’s shoes. You see that LLM’s are getting better and better, and it’s very possible that they will have some impact on jobs in the next few years, and a very meaningful impact on cybersecurity.
Is it more ethical to stay silent about these concerns, as you might have a bit of self interest? Or even if it looks a bit self interested, is it better to warn people ahead of time? I think the latter is obviously the better position.
The issue is both OpenAI and Anthropic have lied so many times that it’s no longer rational to take anything they say at face value.
Also: they don’t have to know they’re lying to say things that aren’t true. There is definitely some cult-like behaviour at the moment on the west coast
I think that Anthropic is fully absolutely unethical. And they lied a lot. They were actively trying to make the doom happen while trying to cash out maximally on doom trolling.
If they were actually concerned over social impact, they would try to minimize it. They could have sell their product as a tool to be used to make economy boom, they tried to sell it on promiss to make it shrink for most people.
It really does not matter how much they believed own doom predictions, because they were actively trying to make them true whether realistic or not.
The point I'm trying to make is Anthropic's marketing about broad security risk related to the capability of its models is a valid concern though their dog and pony show really overdid it, probably to the detriment of us all for many reasons. It is indeed amplifying the abilities of people to find and exploit security issues.
The point of my anecdote is I was able to identify and fix an at least security adjacent bug in a language I could charitably consider myself a novice in. It happened to very unlikely have a security impact, but that was mere chance. LLMs expand the pool of people able to find and exploit security problems and we're all considerably more vulnerable as a result.
The biggest security threat was always someone bored with $20, a lot of attacks could be ignored or at least not prioritized with that threat model. This isn't true any more and our attack surface has gotten a whole lot larger.
Serious question: do you think the NSA aren't training their own LLMs? (With or without Anthropic and OpenAI's help)
It's a perfect technology for their uses, they get a big chunk of a $100 billion black budget, and they've had access to the research for at least as long as we have.
> Serious question: do you think the NSA aren't training their own LLMs?
Given the evergreen discussion of "are these companies making a profit"*, I think any LLMs that the NSA (or any other government agency worldwide) may be making are quite far from the leading edge.
* Person A: "they are making a loss!" Person B: "Only if you count training, they make a profit on inference, look at what it costs to run comparable open models on generic cloud servers" A: "Sure, but if they don't train new models they'll be left behind, so they're still making a loss"
That and the way compute is now measured in GW, I think even random low budget vloggers just getting started would be able to spot if the NSA was doing anything significant just from the extra heat emissions or power plants getting built.
The rate of inference compute to training compute is ~10:1, for popular frontier models. Models are routinely overtrained past the Chinchilla optimum now because it makes an immense amount of economic sense to do so.
Worse the more niche and unused your models get, but when this "making a loss" fuckery pops up, it's usually about the big guys like Anthropic, OpenAI, GDM and maybe xAI and Meta. Of which only the latter can be accused of not selling enough inference to offset the training runs.
The real money sinks are: R&D and infrastructure buildouts.
I don't think there is much overlap between people capable of building cutting edge LLM's and the people who want to build a cutting edge LLM for the government.
The NSA managed to deliberately insert a backdoor into elliptic-curve cryptography right under the noses of everyone capable of making elliptic-curve cryptography.
Mathematicians in academia are paid a little less than AI researchers. Companies are willing to pay billions to steal the few people capable of driving development of frontier LLMs from each other. Cryptographers don't quite enjoy the same popularity.
I can't say what they're doing now because I worked for the NSA 15 years ago but the view of them as an omnipotent power is a product of Hollywood. The government is good at throwing an ungodly amount of resources at something to get a result, and so they are often the source of original development of technologies. The private sector has always been much better at building a technology to greater sophistication and efficiency. There may be blue badgers in Fort Meade trying to train models but there is no chance they are competitive with the frontier AI companies. It's like saying the government has an amazing home-grown fighter aircraft that is beyond what Lockheed has ever made...they delegate that stuff to private companies for a reason.
The NSA is government agency. They are certainly not training any world class LLMs. They probably have some specialized fine tunings of existing models, but that's it. They don't have the capacity.
Serious question, do you realize that the NSA are mere mortals? Do you realize how much it takes to train a model? Does the NSA make their own chips or planes? The NSA buys a lot of technology because they can't make their own.
You mean "Rhetorical question," and I didn't need patronising.
They have at least one pretty vast, largely classified data centre in Utah, with a sizeable chunk of the black budget and they also have pretty large data sets.
NSA has had their own supercomputing program for decades. they design and produce their own large scale machines. chips, fabrics, arithmetic units, all of it. they also employ quite a number of hardcore mathematicians, computer scientists, and systems wranglers. if they decided it was of strategic importance there is absolutely no reason they couldn't train their own models.
If they wanted to officially take the weights the DPA would work and Dario could do nothing. If they wanted to do it in clandestine manner no one could stop them and no one would know. It's very likely they already have all the weights from all the frontier models. I mean all the frontier models are capable of being served from AWS Bedrock so the weights aren't exactly locked in some air-gapped vault.
It would be easy to make a national security justification to take the weights in a clandestine manner especially because Anthropic supposedly got caught giving China access to the model through a cutout.
Pretty sure even under DPA, taking without fair compensation would be a violation of the takings clause of 5th Amendment and wouldn't withstand legal scrutiny. If they wanted to get them clandestinely, yeah, they'd likely get away with it, but it is stealing.
Which is a fundamental mistake to make with the U.S. government, even if we’re talking only about the executive branch, even if we’re only talking about DoD, even if we’re only talking about the IC.
The current position seems to be no-one has access, not even Anthropic employees. What powers does the US government have to force them to provide access? If they have that power why did they not use it to force them to provide their products for military use?
You misunderstand - the government issued a directive to Anthropic that effectively forced them to pull access from everyone, even their own employees.
The directive was to remove access to non-Americans, not to pull access from everyone. It’s because Anthropic cannot verify the identity of its users that it pulled access from everyone, not because the government explicitely requested that.
If their operation team is not US based that's going to be difficult to operate. They would have to reorganize the whole company as I'm pretty sure that they are not employing only US citizen.
Well, kinda in effect. He lacked the authority to make the call, and it is quite obviously being ignored by most suppliers. If it were actually enforced, I believe no companies that are contractors for the DoD could even host Anthropic- like GCP, Azure, and AWS. Perhaps they are currently figuring out how to get off all cloud provider govclouds, but I doubt it.
Probably not. The US constitution limits what government can force on the people. If the NSA tries to force something that will spend years in court (if anyone wants to fight)
> The US constitution limits what government can force on the people.
The US constitution also prohibits:
- refusing to spend money that congress has appropriated
- dismantling congressionally-created federal agencies without congressional authorization
- directing federal agencies to selectively apply the law according to the preference of the executive
- giving control of federal agencies to individuals who have not been appointed by the legislative branch
- terminating, detaining, or deporting people without due process
- retaliation against private citizens or corporations for speech protected under the first amendment
- discriminating on protected grounds under the equal protections clause
... and yet the administration has done all these things with impunity while effete judges wring their hands and write sternly-worded letters. The US constitution demonstrably no longer has any force or effect.
Yeah... NSA literally has MITM proxies/interception of any traffic they want inside every major US tech company (based on my reading/following of Snowden leaks and others). Anthropic wouldn't be able to exist without implicit NSA approval. This article reads more like a marketing piece for Anthropic/Mythos... and ends by talking about how much NSA wants Anthropic models.
You just intercept the traffic after its decrypted on the server side, or are you suggesting you somehow send encrypted traffic that never gets decrypted?
It's just not technically feasible, so there's nothing to lie about. They're not MITMing petabytes/sec across dozens (hundreds?) of companies and they haven't broken TLS1.3.
If I have a box at Digital Ocean and I'm communicating with it with TLS1.3 using a Let's Encrypt cert that I generated, where, exactly, does this magical MITM box come into play?
Of course it's feasible, you just intercept the traffic post-decryption on the cloud/server side. You don't control how/where your traffic to 3p cloud services is decrypted.
> This was their third office space, serving as their headquarters before they outgrew it and eventually relocated to Market Square at 1355 Market Street in 2012. The arab spring twitter uprising was fully a CIA/NSA operation.
To be clear, the claim you're making is that because Twitter has their third corporate office in the same building as an AT&T switching center, and US intelligence used a room in AT&T's switching center for surveillance, then Twitter must have been controlled by US intelligence? And thus the Arab Spring uprising, where Twitter was used, was "fully a CIA/NSA operation"?
Yes, twitter was used by US 3 letter agencies to assist in the arab spring. To be able to do it in a surreptitious way they were asked to move to that building and get access to all private DMs, and for doing so they got a fat tax break to move to "blighted" market street current location. All of those things fit the timeline and snowden capability disclosures.
Yes, you have collected a lot of random bits of information from over a decade ago. I'm sure everything you say is still relevant today, especially the conspiracy nonsense.
Some of us actually work in security, while others think the NSA and CIA are some magically powerful orgs.
Explain how, even with the mystical Room 641A, the NSA can't break a TLS1.3 protected communication channel without either party knowing about it. Assume you have generated a cert with Let's Encrypt. How, exactly, does that work?
Explain to me how you are going to encrypt your LLM API calls with your let's encrypt cert.
There are also multiple ways/places traffic you send to typical cloud/tech company is decrypted and can be intercepted. (Surprised I have to point this out to someone who 'actually works in security ' lol)
Not to mention US tech companies fully cooperate with the NSA in many cases and are aware of this going on.
why is europe going to such great lengths to build datacenters and ensure they have no connection to US jurisdiction... GDPR means nothing if there is a persistent threat installed on every instance.
you compel the host under similar threat of non-existance to grant you view of the hypervisor. you're not running on bare metal with alternate TPM's that arent the Intel IMU (also backdoor'd) so you're just as pwnable.
now say you're doing this on a raspberry pi or other openhardware like a librum machine with a yubikey hsm on local wifi or physical ethernet... you may have a shot at the privacy you're looking for.
I worked on these cases at EFF and I'm skeptical of the automatic "NSA has access to everything" intuition.
What we learned from that era includes things like
(1) spy agencies are incredibly aggressive and pursue tons of different angles to get access to things
(2) spy agencies have a lot of money
(3) spy agencies often have interpretations of law that would surprise the public or legal experts (and sometimes courts have issued sealed rulings permitting them to do things that surprise the public or legal experts later when they're unsealed)
(4) some people throughout different parts of society assume culturally that companies in a country "should" generally help the spy agencies of that country's government because they are the "good guys" or "on the same team" or whatever
These things are all pretty bad and scary, but they still don't imply absolutely infinite power or access, because all of them come with different kinds of pushback. People also just tell them no!
I want to write an article with a colleague about the continuing role of culture here, because I think there are companies or industries where the default reaction is to want to cooperate with the government, and others where the default reaction is not that.
There are certainly secret things that have never come out, e.g. whatever Senator Wyden keeps alluding to, and what kind of program or authority was behind the interception of hardware shipments to covertly tamper with them, and whether there is a bulk financial data interception program, and presumably lots of other stuff. I don't agree with these things, and I want them to be exposed and stopped, and I also don't think they constitute infinite power over all parts of the tech industry.
>they're cops with too much access, it doesn't take a genius to outsmart a cop
the nsa has an unlimited budget and spend a good portion of that budget recruiting some of the smartest people in the country. while they dont have super powers, they also arent the town cop who took a 6 month course after high school then joined the force.
it does no good to hold them up as mythical figures. it also does no good to pretend they are bumbling idiots.
(every math phd i am acquainted with has been approached by nsa recruiters. none of them have been approached by police agencies.)
Some of the smartest people I know have worked on fighting NSA, but they had a drastically smaller budget than NSA itself, and the mental availability bias is skewed by the fact that the "fighting NSA" people talked about their work all the time, while the "being NSA" people generally didn't.
I do know one extremely smart person who went to work there, and I witnessed a failed recruitment of another extremely smart person.
> every math phd i am acquainted with has been approached by nsa recruiters.
how many of them took them up on the offer, and how many are in leadership roles?
it takes a very narrow range of personality to want to be a cop, which at the end of the day is a government job... the only people they make rich are contractors
I'm not saying there aren't smart people working there but it's ridiculous to assume they have an iron grasp on all communication from the top tech companies in the world, while also monitoring half the world's governments... they just don't
>how many of them took them up on the offer, and how many are in leadership roles?
this is not really relevant to the point, but to satisfy your curiosity: more than one, and one.
>it takes a very narrow range of personality to want to be a cop
the nsa's brightest aren't doing "cop" things. certainly none of the people i know of working there are "cop-minded" in any sense.
they are doing cool research and application things. otherwise they wouldn't be able to entice the phds to stick around. these are people that want to work at the forefront of their field, doing interesting work, and the nsa is one avenue of doing that (with good job security, benefits, etc.).
>it's ridiculous to assume they have an iron grasp on all communication from the top tech companies in the world, while also monitoring half the world's governments
we agree here. they are certainly doing "HNDL" (harvest now, decrypt later) at a very large scale. but obviously they are not able to collect and store every piece of communication at every tech company over years and years. (the intelligence community comprehensive national cybersecurity initiative data center is large, but not that large)
> how many of them took them up on the offer, and how many are in leadership roles?
In my cohort? Several, and who knows? The recruitment effort is very visible and intense.
The US math PhD market has been a slow-rolling disaster for over a decade. Everyone who can hack it outside the ivory tower is actively looking for the exits.
So why is it surprising that some of them go to work at the NSA?
If Mythos is still running internally, the NSA still have some access to it. It's just crazy to believe there aren't CIA and/or NSA plants (tacitly acknowledged or otherwise) inside Anthropic and OpenAI.
But Mythos is still only an advanced LLM so I am not sure what all this breathy fuss is about; it sounds like the PR war more than anything.
If the NSA aren't themselves training technologies that are at least as powerful, that would modestly surprise me.
Not that you need an LLM to monitor the risks to the USA. You just need Tulsi Gabbard's emails.
I think it’s beyond a mastery of PR. They literally called it Mythos and built a literal myth around it. I mean… maybe people just want the soap opera.
NSA has produced nothing. Does nothing. Why don't we just have them pick up garbage on the freeway to help out the tax payers? Let Anthropic and other adults push spying forward.
Have we become reddit here? I mean, you probably have some sympathy and upvotes from fellow readers, but this isn't pushing the conversation forward at all.
Any citations to your statement that NSA produces nothing? Or do you have a strong argument or evidence to support this?
Kind of crazy actually. Other models are catching up fast, they all can find the vulnerabilities in our (and by our I mean everyone's) underlying infra very fast. It takes a very long time to fix, review, and finally deploy these fixes. There really isn't much time left.
mythos allowed mediocre people to get results by holding their hand through the process, or just ignoring their irrelevant input and knowing what to do.
if you throw millions of tokens at IDA Pro MCP with the right prompt lets just say security by obscurity fails miserably because there is no obscurity when the LLM chews through the decompilation.
It isn’t bad, it isn’t good. It’s just how the world looks now. All software is open source now, some of it is just more open, some of it is less.
they are doing DOGE-cuts to all of intelligence now anyway
dozens upon dozens fired for no reason
so US "intelligence" is going to go even further backwards
* https://www.yahoo.com/news/politics/articles/trump-acting-ch...
November is going to be insanity
The NSA is managed by the NSA director, an independently appointed and confirmed office separate from the Office of the Director of National Intelligence. The DNI does have the authority to restrict funds to the NSA, and sets certain priorities, policy, etc. but the NSA director is not beholden to the DNI and makes their hiring and firing decisions independently. They’re also, currently and historically though not required by statute, a flag officer in the US military and dual-hatted as the commander of CYBERCOM. All this is to say, chaos in the office of the DNI does not necessarily impact the NSA.
https://www.cbsnews.com/news/tim-haugh-firing-trump-60-minut...
> Haugh was fired in April of this year after far-right activist Laura Loomer met with President Trump.
What kind of sick joke is that
Here is one sector of the US government I'm happy to see burned down. If the alternative is the status quo, I'm OK with any roll of the dice.
It can always get worse.
Pah! Nonsense. What could possibly be worse than Weimar Germany or Tsarist Russia?
Speed running through that to 1984?
You're assuming that they are "dismantling" it in a sense, what they are actually doing is mostly attacking workers while introducing extremely unsafe software. If you think LLMs are terrible, imagine it being the gatekeeper on whether your personal info is shared to an individual (and they ain't wasting time on the boring info either!).
When you want to reorient the government, it's much easier doing it with a smaller more loyal force. Now introduce tools that make mass surveillance easier and less accountable.
Like that's not a bad thing for them, that's what they want to do.
---
Back to the article, I'm not shocked that a massive LLM company speed running into the brick wall that is the US government; just thought it would be OpenAI, but Sam Altman is truly the best bottom feeder the game.
Also fully believe that Anthropic is hoping that public sentiment is on their side but more Americans hate AI companies than Trump so it's not going to go how they want.
Give it maybe 3-6 months before the Trump Admin talks about openly nationalizing Anthropic.
Anecdata suggests NSA just got on board and kept going tbh. Not sure they’ve felt the same impacts / churn as other agencies, and not sure they’ve ever really been that beholden to the DNI.
> dozens upon dozens fired for no reason
When you say without reason do you mean without cause?
Seems to me OP's implication is that they were fired because someone wanted to hit a quota of (employees cut/payroll expenses reduced), or other similarly ''reasonless'' justifications.
AI marketing bullshit stunts are unlike anything I've seen in 30 years. It started with MS Copilot so called capabilities for work, which were completely made up use cases that didn't work at all (3 years later still). We've had OpenAI "AGI is coming" and "AI will take your job", now we have Mythos being so "dangerous" for cybersecurity, which of course makes the average Joe interpret it as Anthropic being "the better overall company, the NSA uses it!!". I mean gov foes with Anthropic are probably true, but the marketing is to blame not Mythos capabilities. This is all so fucking pathetic
> and "AI will take your job"
Don't forget, its no longer cool to say that now that the public has pushed back. The fact they all changed their tone away from taking jobs tells you that it was all just entirely marketing.
All the CEOs very quickly changed their messaging after Altman's house got molotoved.
Seems to me that they were mostly right, and the message was received by the right people. No need to ensure it gets distributed to the wrong people.
But the propaganda deluge was a smash hit so far, HN is drowning in “AI” BS, and astroturfers and spin doctors haven’t seen that much business since the cold war. They made more profit than shovel salesmen in the gold rush.
We should seriously reframe this whole AI thing to "SI = simulated intelligence".
It's google in a box. Great achievement, makes knowledge work faster, but please stop bothering everyone else.
The Uber and Groupon people became billionaires, so the "Simulated Intelligence" folks will also achieve it. No need to worry and drown everyone in these bs stories only non-tech people believe.
Heh. In the Schlock Mercenary universe, "SI" means "synthetic intelligence", which is a level below real AI (which means what we would call AGI). And, as it says (in https://www.schlockmercenary.com/2003-07-21), SI translates to "kinda stupid".
I was able to identify, diagnose, fix, and upstream a minor bug in and erlang/OTP ssh key implementation with Opus in maybe 20 minutes (+2 weeks or so for upstream). It is not impossible that I could have done this before, but it would have taken days or weeks. The actual fix was about 2 lines of code, hardly AI slop, but getting there would have been quite the slog, and I never would have done it.
There is a lot of the reason for AI skepticism out there, but people tend to do massive overcorrections and underestimate the force multiplier it can be, particularly for people with some idea of what they're doing and a good grasp of how to take advantage of the tool.
I said absolutely nothing about LLMs, which is a fantastic tool I'm using every day. I'm talking about marketing.
So let’s say you’re in Anthropic’s shoes. You see that LLM’s are getting better and better, and it’s very possible that they will have some impact on jobs in the next few years, and a very meaningful impact on cybersecurity.
Is it more ethical to stay silent about these concerns, as you might have a bit of self interest? Or even if it looks a bit self interested, is it better to warn people ahead of time? I think the latter is obviously the better position.
The issue is both OpenAI and Anthropic have lied so many times that it’s no longer rational to take anything they say at face value.
Also: they don’t have to know they’re lying to say things that aren’t true. There is definitely some cult-like behaviour at the moment on the west coast
I think that Anthropic is fully absolutely unethical. And they lied a lot. They were actively trying to make the doom happen while trying to cash out maximally on doom trolling.
If they were actually concerned over social impact, they would try to minimize it. They could have sell their product as a tool to be used to make economy boom, they tried to sell it on promiss to make it shrink for most people.
It really does not matter how much they believed own doom predictions, because they were actively trying to make them true whether realistic or not.
The point I'm trying to make is Anthropic's marketing about broad security risk related to the capability of its models is a valid concern though their dog and pony show really overdid it, probably to the detriment of us all for many reasons. It is indeed amplifying the abilities of people to find and exploit security issues.
The point of my anecdote is I was able to identify and fix an at least security adjacent bug in a language I could charitably consider myself a novice in. It happened to very unlikely have a security impact, but that was mere chance. LLMs expand the pool of people able to find and exploit security problems and we're all considerably more vulnerable as a result.
The biggest security threat was always someone bored with $20, a lot of attacks could be ignored or at least not prioritized with that threat model. This isn't true any more and our attack surface has gotten a whole lot larger.
All for a product that has yet to make a single honest dollar in profit for anyone who isn't nvidia.
When this goes we might well see a recession. Not that anyone responsible will be worse off, of course.
The perpetrators all have their golden parachutes. The taxpayers will foot the bill.
Why on earth would you expect any of them to take profit so early in the game?
Silly me, expecting a company worth a trillion dollars to make... some money. Any money. A single profitable product.
Everyone lost access. What even is mainstream news these days.
They could easily take the weights if they wanted. I don't believe they meaningfully lost access.
Who will make them the next set of weights?
If a government can just seize the product of someone else's labour, either they will end up as slave owners or without willing workers.
Serious question: do you think the NSA aren't training their own LLMs? (With or without Anthropic and OpenAI's help)
It's a perfect technology for their uses, they get a big chunk of a $100 billion black budget, and they've had access to the research for at least as long as we have.
> Serious question: do you think the NSA aren't training their own LLMs?
Given the evergreen discussion of "are these companies making a profit"*, I think any LLMs that the NSA (or any other government agency worldwide) may be making are quite far from the leading edge.
* Person A: "they are making a loss!" Person B: "Only if you count training, they make a profit on inference, look at what it costs to run comparable open models on generic cloud servers" A: "Sure, but if they don't train new models they'll be left behind, so they're still making a loss"
That and the way compute is now measured in GW, I think even random low budget vloggers just getting started would be able to spot if the NSA was doing anything significant just from the extra heat emissions or power plants getting built.
Model training does NOT dominate the model costs.
The rate of inference compute to training compute is ~10:1, for popular frontier models. Models are routinely overtrained past the Chinchilla optimum now because it makes an immense amount of economic sense to do so.
Worse the more niche and unused your models get, but when this "making a loss" fuckery pops up, it's usually about the big guys like Anthropic, OpenAI, GDM and maybe xAI and Meta. Of which only the latter can be accused of not selling enough inference to offset the training runs.
The real money sinks are: R&D and infrastructure buildouts.
I don't think there is much overlap between people capable of building cutting edge LLM's and the people who want to build a cutting edge LLM for the government.
The NSA managed to deliberately insert a backdoor into elliptic-curve cryptography right under the noses of everyone capable of making elliptic-curve cryptography.
I wouldn't count them out.
Mathematicians in academia are paid a little less than AI researchers. Companies are willing to pay billions to steal the few people capable of driving development of frontier LLMs from each other. Cryptographers don't quite enjoy the same popularity.
I can't say what they're doing now because I worked for the NSA 15 years ago but the view of them as an omnipotent power is a product of Hollywood. The government is good at throwing an ungodly amount of resources at something to get a result, and so they are often the source of original development of technologies. The private sector has always been much better at building a technology to greater sophistication and efficiency. There may be blue badgers in Fort Meade trying to train models but there is no chance they are competitive with the frontier AI companies. It's like saying the government has an amazing home-grown fighter aircraft that is beyond what Lockheed has ever made...they delegate that stuff to private companies for a reason.
The NSA is government agency. They are certainly not training any world class LLMs. They probably have some specialized fine tunings of existing models, but that's it. They don't have the capacity.
You cannot really hide the amount of compute required to train an LLM. Do we have actual clues that NASA is training their own frontier model?
They probably also have an insane dataset
> do you think the NSA aren't training their own LLMs?
They probably already have access to Sentinel, so they wouldn't need to train their own.
Serious question, do you realize that the NSA are mere mortals? Do you realize how much it takes to train a model? Does the NSA make their own chips or planes? The NSA buys a lot of technology because they can't make their own.
You mean "Rhetorical question," and I didn't need patronising.
They have at least one pretty vast, largely classified data centre in Utah, with a sizeable chunk of the black budget and they also have pretty large data sets.
NSA has had their own supercomputing program for decades. they design and produce their own large scale machines. chips, fabrics, arithmetic units, all of it. they also employ quite a number of hardcore mathematicians, computer scientists, and systems wranglers. if they decided it was of strategic importance there is absolutely no reason they couldn't train their own models.
the success of mythos isn't from model weights, it's from the harness and toolset it has access to
I agree but that's even easier to exfiltrate, surely.
given some time, surely. but that seems harder with the model turned off.
Is it really?
Harness is important for model performance, but weights are surely mode important, without that you would have haiku doing the work.
If they use the defence production act, would Dario be even able to resign in protest?
If they wanted to officially take the weights the DPA would work and Dario could do nothing. If they wanted to do it in clandestine manner no one could stop them and no one would know. It's very likely they already have all the weights from all the frontier models. I mean all the frontier models are capable of being served from AWS Bedrock so the weights aren't exactly locked in some air-gapped vault.
It would be easy to make a national security justification to take the weights in a clandestine manner especially because Anthropic supposedly got caught giving China access to the model through a cutout.
Pretty sure even under DPA, taking without fair compensation would be a violation of the takings clause of 5th Amendment and wouldn't withstand legal scrutiny. If they wanted to get them clandestinely, yeah, they'd likely get away with it, but it is stealing.
John Cook resigned, so Dario might resign also. But he would make it public, so they won't do it
> John Cook resigned
John Cook?
He means John Apple I think.
Doesn't make any sense. They could just force them to provide Mythos to the federal government.
> They could just force them to provide Mythos to the federal government
The DPA only gives that power to the President [1].
[1] https://en.wikipedia.org/wiki/Defense_Production_Act_of_1950
Maybe GP was treating Trump to the royal "they"
Which is a fundamental mistake to make with the U.S. government, even if we’re talking only about the executive branch, even if we’re only talking about DoD, even if we’re only talking about the IC.
doubt Trump would accept that pronoun
The current position seems to be no-one has access, not even Anthropic employees. What powers does the US government have to force them to provide access? If they have that power why did they not use it to force them to provide their products for military use?
> What powers does the US government have to force them to provide access?
https://en.wikipedia.org/wiki/Defense_Production_Act_of_1950
This would not be a particularly big stretch here, either.
There is even a precedent for its use with regard to AI (only disclosing information, but still). Biden used it, why does Trump not do so?
You misunderstand - the government issued a directive to Anthropic that effectively forced them to pull access from everyone, even their own employees.
The directive was to remove access to non-Americans, not to pull access from everyone. It’s because Anthropic cannot verify the identity of its users that it pulled access from everyone, not because the government explicitely requested that.
If their operation team is not US based that's going to be difficult to operate. They would have to reorganize the whole company as I'm pretty sure that they are not employing only US citizen.
>I'm pretty sure that they are not employing only US citizen
Understatement. They have 14 offices, only 4 of them are in the US (6 are in EMEA, 4 in APAC).
> directive was to remove access to non-Americans
Did Hegseth pull his supply-chain risk BS?
No - this was a separate power unrelated to the supply chain risk which is still in effect.
Well, kinda in effect. He lacked the authority to make the call, and it is quite obviously being ignored by most suppliers. If it were actually enforced, I believe no companies that are contractors for the DoD could even host Anthropic- like GCP, Azure, and AWS. Perhaps they are currently figuring out how to get off all cloud provider govclouds, but I doubt it.
Probably not. The US constitution limits what government can force on the people. If the NSA tries to force something that will spend years in court (if anyone wants to fight)
The constitution limits a lot of things that this administration has done regardless.
> The US constitution limits what government can force on the people.
The US constitution also prohibits:
- refusing to spend money that congress has appropriated
- dismantling congressionally-created federal agencies without congressional authorization
- directing federal agencies to selectively apply the law according to the preference of the executive
- giving control of federal agencies to individuals who have not been appointed by the legislative branch
- terminating, detaining, or deporting people without due process
- retaliation against private citizens or corporations for speech protected under the first amendment
- discriminating on protected grounds under the equal protections clause
... and yet the administration has done all these things with impunity while effete judges wring their hands and write sternly-worded letters. The US constitution demonstrably no longer has any force or effect.
Yeah... NSA literally has MITM proxies/interception of any traffic they want inside every major US tech company (based on my reading/following of Snowden leaks and others). Anthropic wouldn't be able to exist without implicit NSA approval. This article reads more like a marketing piece for Anthropic/Mythos... and ends by talking about how much NSA wants Anthropic models.
Propaganda.
> NSA literally has MITM proxies/interception of any traffic they want inside every major US tech company
No, they don't.
https://blog.encrypt.me/2013/11/05/ssl-added-and-removed-her...
https://en.wikipedia.org/wiki/Room_641A
Yeah, they did (and probably do).
How are they going to MITM communications with certs that never left my machine?
Are you suggesting they broke TLS or that they've somehow acquired every private cert generated?
You just intercept the traffic after its decrypted on the server side, or are you suggesting you somehow send encrypted traffic that never gets decrypted?
It's back to the question of how much you should give the benefit of doubt to powerful people who openly lie.
It's just not technically feasible, so there's nothing to lie about. They're not MITMing petabytes/sec across dozens (hundreds?) of companies and they haven't broken TLS1.3.
If I have a box at Digital Ocean and I'm communicating with it with TLS1.3 using a Let's Encrypt cert that I generated, where, exactly, does this magical MITM box come into play?
Of course it's feasible, you just intercept the traffic post-decryption on the cloud/server side. You don't control how/where your traffic to 3p cloud services is decrypted.
That "box" is a virtual machine, no?
Do you know what hypervisor is managing it? :)
... not your machines, not your crypto...
[flagged]
> This was their third office space, serving as their headquarters before they outgrew it and eventually relocated to Market Square at 1355 Market Street in 2012. The arab spring twitter uprising was fully a CIA/NSA operation.
To be clear, the claim you're making is that because Twitter has their third corporate office in the same building as an AT&T switching center, and US intelligence used a room in AT&T's switching center for surveillance, then Twitter must have been controlled by US intelligence? And thus the Arab Spring uprising, where Twitter was used, was "fully a CIA/NSA operation"?
Yes, twitter was used by US 3 letter agencies to assist in the arab spring. To be able to do it in a surreptitious way they were asked to move to that building and get access to all private DMs, and for doing so they got a fat tax break to move to "blighted" market street current location. All of those things fit the timeline and snowden capability disclosures.
The CIA venture arm InQTel invested in Dataminr a company that twitter was also a major shareholder. https://theintercept.com/2016/04/14/in-undisclosed-cia-inves...
Yes, you have collected a lot of random bits of information from over a decade ago. I'm sure everything you say is still relevant today, especially the conspiracy nonsense.
Some of us actually work in security, while others think the NSA and CIA are some magically powerful orgs.
Explain how, even with the mystical Room 641A, the NSA can't break a TLS1.3 protected communication channel without either party knowing about it. Assume you have generated a cert with Let's Encrypt. How, exactly, does that work?
Explain to me how you are going to encrypt your LLM API calls with your let's encrypt cert.
There are also multiple ways/places traffic you send to typical cloud/tech company is decrypted and can be intercepted. (Surprised I have to point this out to someone who 'actually works in security ' lol)
Not to mention US tech companies fully cooperate with the NSA in many cases and are aware of this going on.
why is europe going to such great lengths to build datacenters and ensure they have no connection to US jurisdiction... GDPR means nothing if there is a persistent threat installed on every instance.
you compel the host under similar threat of non-existance to grant you view of the hypervisor. you're not running on bare metal with alternate TPM's that arent the Intel IMU (also backdoor'd) so you're just as pwnable.
now say you're doing this on a raspberry pi or other openhardware like a librum machine with a yubikey hsm on local wifi or physical ethernet... you may have a shot at the privacy you're looking for.
Thank you.
Please provide sources for such bold claims
https://en.wikipedia.org/wiki/PRISM
https://www.wired.com/2013/10/nsa-hacked-yahoo-google-cables...
https://www.eff.org/nsa-spying
I worked on these cases at EFF and I'm skeptical of the automatic "NSA has access to everything" intuition.
What we learned from that era includes things like
(1) spy agencies are incredibly aggressive and pursue tons of different angles to get access to things
(2) spy agencies have a lot of money
(3) spy agencies often have interpretations of law that would surprise the public or legal experts (and sometimes courts have issued sealed rulings permitting them to do things that surprise the public or legal experts later when they're unsealed)
(4) some people throughout different parts of society assume culturally that companies in a country "should" generally help the spy agencies of that country's government because they are the "good guys" or "on the same team" or whatever
These things are all pretty bad and scary, but they still don't imply absolutely infinite power or access, because all of them come with different kinds of pushback. People also just tell them no!
I want to write an article with a colleague about the continuing role of culture here, because I think there are companies or industries where the default reaction is to want to cooperate with the government, and others where the default reaction is not that.
There are certainly secret things that have never come out, e.g. whatever Senator Wyden keeps alluding to, and what kind of program or authority was behind the interception of hardware shipments to covertly tamper with them, and whether there is a bulk financial data interception program, and presumably lots of other stuff. I don't agree with these things, and I want them to be exposed and stopped, and I also don't think they constitute infinite power over all parts of the tech industry.
the NSA isn't a bunch of super soldiers, they're cops with too much access, it doesn't take a genius to outsmart a cop
>they're cops with too much access, it doesn't take a genius to outsmart a cop
the nsa has an unlimited budget and spend a good portion of that budget recruiting some of the smartest people in the country. while they dont have super powers, they also arent the town cop who took a 6 month course after high school then joined the force.
it does no good to hold them up as mythical figures. it also does no good to pretend they are bumbling idiots.
(every math phd i am acquainted with has been approached by nsa recruiters. none of them have been approached by police agencies.)
I appreciate the balance here.
Some of the smartest people I know have worked on fighting NSA, but they had a drastically smaller budget than NSA itself, and the mental availability bias is skewed by the fact that the "fighting NSA" people talked about their work all the time, while the "being NSA" people generally didn't.
I do know one extremely smart person who went to work there, and I witnessed a failed recruitment of another extremely smart person.
> every math phd i am acquainted with has been approached by nsa recruiters.
how many of them took them up on the offer, and how many are in leadership roles?
it takes a very narrow range of personality to want to be a cop, which at the end of the day is a government job... the only people they make rich are contractors
I'm not saying there aren't smart people working there but it's ridiculous to assume they have an iron grasp on all communication from the top tech companies in the world, while also monitoring half the world's governments... they just don't
>how many of them took them up on the offer, and how many are in leadership roles?
this is not really relevant to the point, but to satisfy your curiosity: more than one, and one.
>it takes a very narrow range of personality to want to be a cop
the nsa's brightest aren't doing "cop" things. certainly none of the people i know of working there are "cop-minded" in any sense.
they are doing cool research and application things. otherwise they wouldn't be able to entice the phds to stick around. these are people that want to work at the forefront of their field, doing interesting work, and the nsa is one avenue of doing that (with good job security, benefits, etc.).
>it's ridiculous to assume they have an iron grasp on all communication from the top tech companies in the world, while also monitoring half the world's governments
we agree here. they are certainly doing "HNDL" (harvest now, decrypt later) at a very large scale. but obviously they are not able to collect and store every piece of communication at every tech company over years and years. (the intelligence community comprehensive national cybersecurity initiative data center is large, but not that large)
> how many of them took them up on the offer, and how many are in leadership roles?
In my cohort? Several, and who knows? The recruitment effort is very visible and intense.
The US math PhD market has been a slow-rolling disaster for over a decade. Everyone who can hack it outside the ivory tower is actively looking for the exits.
So why is it surprising that some of them go to work at the NSA?
> Propaganda
IPO incoming.
Misdirection
If Mythos is still running internally, the NSA still have some access to it. It's just crazy to believe there aren't CIA and/or NSA plants (tacitly acknowledged or otherwise) inside Anthropic and OpenAI.
But Mythos is still only an advanced LLM so I am not sure what all this breathy fuss is about; it sounds like the PR war more than anything.
If the NSA aren't themselves training technologies that are at least as powerful, that would modestly surprise me.
Not that you need an LLM to monitor the risks to the USA. You just need Tulsi Gabbard's emails.
I think it’s beyond a mastery of PR. They literally called it Mythos and built a literal myth around it. I mean… maybe people just want the soap opera.
NSA has produced nothing. Does nothing. Why don't we just have them pick up garbage on the freeway to help out the tax payers? Let Anthropic and other adults push spying forward.
Have we become reddit here? I mean, you probably have some sympathy and upvotes from fellow readers, but this isn't pushing the conversation forward at all.
Any citations to your statement that NSA produces nothing? Or do you have a strong argument or evidence to support this?
They're great at building datacenters and running massive archival operations.
Ghidra comes to mind
https://github.com/nationalsecurityagency/ghidra
Kind of crazy actually. Other models are catching up fast, they all can find the vulnerabilities in our (and by our I mean everyone's) underlying infra very fast. It takes a very long time to fix, review, and finally deploy these fixes. There really isn't much time left.