Chinese AI companies are free to use any Nvidia GPU they want as long as the GPU is not physically located within China. It is entirely consistent with US export controls for Chinese AI companies to remotely use GPUs located in Singapore or Malaysia.
“In facilities across the region, Megaspeed rents Nvidia chips to Chinese tech giants such as Alibaba, an arrangement that does not violate current US export control regulations.” See [1].
Literally my first thought. Noone can stop them from training in data centers that are located elsewhere. It's not like some military hardware that needs to be physically shipped to China for them to make use of it. In fact why would they bother with Chinese compute providers if those are limited because of GPU export controls? They knew this for a long time and probably have been knocking on the door of providers in other countries for a while. Heck, they could be training inside the US. They already distribute their open model weights from the US.
The US definitely has the power to stop most of it. It just looks like the executive branch doesn’t yet have the legal authority to do it, but it’s in an advanced legislative state.
How are you going to monitor who is logging into a system, or who a job is being run for? If you want that level of control the machines should not be exported at all.
Datacenters are very large and not very mobile and full of very expensive export controlled equipment and training runs are large expensive contracts. This is not exactly a difficult context to enforce KYC rules in
Same way they do for Iran: if they catch you providing services to someone in Iran, they send you to jail for life. And they monitor all financial transactions so they'll notice.
Yeah but if you provide any service to someone in Iran it’s illegal. Meanwhile there are plenty of legitimate services that can be provided to China. That’s the difficulty. You can’t just monitor financial transactions because you don’t what the transaction is for.
They can do that to Americans. Harder for the rest of the world. How is the US going to monitor CIPS payments? How will they jail someone who never goes to the US? Just how much global jurisdiction is the US claiming?
In practice, the key ways that the US exerts influence outside of its own borders: (1) Sanction the company, which means any other company that does business with that company can also be sanctioned. (2) If enough companies are evading sanctions in a single country or the govt is helping those companies, then threaten to cut the country from USD settlement with the Federal Reserve, e.g., no USD FX allowed. That pretty much screws most economies. The "long arm" of the US govt is very, very, very long!
To be clear: I am not saying any of this is fair (or legal); I am only explaining how it works.
> How will they jail someone who never goes to the US?
The same way they "jail" UN-appointed judges that go against their masters' interests: de-platform them from all US-controlled payment and software platforms.
Yes, they can. They cannot buy things from Amazon or book through booking.com, as that French judge said, but they can do things that do not require American systems. Do you have any evidence they cannot have phones? What stops they shopping in person? Or booking directly with a hotel?
> He said: “We are going 30 years back in time. It’s like a time machine returning us to a pre-digital world,” he told France TV.
Considering the fact that booking.com is Dutch/European that means that they can’t do business with any company that has a significant presence in the US. Presumably that would also affect major hotel chains?
Sanctions only become effective if they are targeted well enough to ensure compliance. If you start making them too broad they will fail, as happened in Russia.
So another company in Asia is going to execute the batch and send the result back to China. And you can add as many indirection levels as you like. This is going to be impossible to track.
You'd just invert it. Make it so you have to be approved under export controls as a customer like they were gonna do for mythos (and like how mythos sort of still works) and make it an allowlist instead of a denylist
If that company is acting in defiance of sanctions, like any other company that acts as a middleman to circumvent sanctions, and it is caught, it will face severe consequences.
There's no gotcha here, companies the world over could just ignore any sanctions they wanted, and some try to, but they do so knowing they themselves are (potentially) breaking local and international laws.
You might have well asked "what if a company helps you by importing contraband instead of you doing it directly"
One of the key reasons that a country like the USA can enforce sanctions is that it can tell other countries that if their companies/citizens/etc break the sanctions, then the USA will put restrictions on those countries ability to trade with the USA, which is why only a few countries will trade with Iran, North Korea, and Russia
At some point those countries will also tire of the USA trying to tell them what to do and will tell the USA politely to fuck off. This is basically what India did with US sanctions on Russia and Iran.
Given the USA has completely torched its reputation amongst allies in the space of a year, I would expect this to happen more often.
Exactly right. The term I use is "economic network effect".
Does anyone know why does the US allows Canada to trade with Cuba? That one always stood out to me as a bit odd. My guess: Canada is the largest trade partner with the US, so the US probably has more to lose by sanctioning Canada.
The US can certainly tell other countries things. And they can even pretend to listen. Or not, as in the case of India and China's response to oil sanctions.
Yes, that's called a treaty, or a trade agreement. Remember all the fuss about Investor/State Dispute Settlement clauses? Most countries have them with each other by now.
To add to this, if the provider suspects that a customer is facilitating sanction busting, or could reasonably be expected to have foreseen it, that's enough for a discussion with lawyers...
> sanctions have successfully applied to countries such as Russia, Iran, etc for decades.
Largely because other countries are willing to go along with it, and to the extent they are willing to do so.
Even NATO countries import Russian gas. The UK rolled back sanctions on Russia to import diesel earlier this year. Most of the rest of the world did not impose effective sanctions and lots of people trade with them (including huge economies like China).
The rest of the world is to dependent on China to want to get involved. The US might succeed in the current administrations attempt to split the world into new cold war spheres of influence (because people are scared of China too) but its not going to be easy, it will be painful, and its not going to get many countries to completely agree to what the US wants.
China is not Russia or Iran it has the means to retaliate economically and politically if pushed to far. Even sanctioning Russia required great effort and huge amounts of self-sacrifice in Europe (are American consumers willing to tolerate that?).
You're going to make it the provider's problem, unleash the full wrath of the US Government on one or two who don't comply pour encourager les autres, and then it self-regulates(ish).
I was wondering why you wrote this part in French instead of English. Turns out it is a phrase that is sometimes loaned into English. It comes from the (French) writings of Voltaire.
Or our allies that we control. But in terms of how - remember the mythos launch? Im absolutely not saying it's necessarily a good idea, I'm just that it's not that hard if we want to do it.
Mythos is a closed model from a US company trained and deployed in the US. The article says they have no idea where and how Chinese models were trained, only that they used GPUs that can't be sold to China. Apart from sanctioning anyone who does business with China (which would probably hurt the US more than anyone) they can't really do anything about that.
They're also free to use Nvidia GPUs physically located in China. However, customs and border enforcement will try to stop them from moving GPUs physically located outside China to physical locations inside China. However, if they can bypass that, they can use the GPU.
Is there any international regulation that forces China to comply with arbitrary US export controls? If not I can't see why Chinese companies are not free to use any Nvidia GPU they can get their hands on period (How the US would then react towards China, the Chinese company and Nvidia is another question).
"A key player in getting Megaspeed off the ground was Alibaba, the Chinese e-commerce leader that has also been an aggressive player in AI, said people familiar with the negotiations at the time. The U.S. had begun imposing export controls on advanced chips in late 2022, preventing Nvidia from selling them directly to China over concerns they could boost China’s military and surveillance capabilities.
"Alibaba was looking for a partner that could help it access computing power from data centers stuffed with Nvidia chips located outside China, the people said."
The author of this piece is a SWE-turned-influencer "China hawk" from OpenAI (i.e. the primary advocate of export bans along with Anthropic), and this isn't an academic paper despite the theming.
I have no idea whether this is legit or not (there's no link to any code), I'm mentioning this because all the comments here are assuming truth without verifying the contents.
The US could impose secondary sanctions to close the loophole, or pressure these countries behind closed doors to not allow these data center uses. Since the only goal of the export restrictions is to hamper Chinese AI efforts, that would be a logical step. If there is one thing that is certain then it's that the current US administration doesn't care at all about open trade and international trade agreements.
The method seems sound and the findings correct. But I think many are missing that (as the research itself says) this doesn't prove any illegal behavior. It's perfectly legal for Chinese companies to train models on GPUs that they rent outside of China. It's also legal for companies to buy GPUs and install them in Malaysia with the intention of renting them to Chinese companies.
There isn't even an allegation that the Chinese companies lied about this. They simply didn't disclose which chips were used. In all cases where they did disclose the training chips, this research found the disclosures to be accurate.
The small print in TFA states the many limitations of their method, i.e. that it actually provides no evidence about which GPUs were really used for training, and much less about who was their owner and where they were located.
The method just strongly suggests that the initial values of the weights, before starting training, were generated with the NVIDIA cuRAND library on a Blackwell GPU (or on an AMD GPU, where the NVIDIA library is emulated for compatibility).
The training might have been done with thousands of GPUs of other types. It is plausible that the GPUs used for training or at least a part of them may have been of the same type with the one used for initializing the model, but it is not proven in any way.
A scenario that matches what is reported in the article, but in which there is only a single Blackwell GPU involved, is if the LLM was intialized on the workstation of some lead developer, who had the privilege to use the best GPU they had, and then it was transferred to a server cluster for training, where other kinds of GPUs were used.
Now that this has been published, it would be very simple in the future for any Chinese AI company to avoid this "fingerprinting", by just using a different PRNG, instead of cuRAND.
Yes it's simple to avoid this fingerprinting now that people know about it, but nobody knew about it before. I don't believe people are casually transferring snapshots between chip architectures without a really good reason, either. It's hard enough to coordinate these training runs without involving multiple chip architectures in a single run.
'Legal' and 'illegal' apply to Nvidia or other US companies facilitating these activities. Perfectly legal for Nvidia to supply GPUs to be rented by Chinese companies, or for Oracle to build datacenters for Chinese companies to rent, as long as they are installed outside of China.
You'd think so but the USA is also the country that's most advanced at getting its legal system to apply outside of it. E.g. they shut down the International Criminal Court.
Indeed, but the USA has been flexing it's muscles against small countries all over the world for decades. However, go back and study the Korean war and Vietnam war, where China and even Russia intervened, and the US lost their arses and was sent packing both times. My eldest uncle was an officer/pilot in the USAF involved in the Vietnam war, flew many bombing missions. What do we have to show, for both those conflicts ?
I used to think the same, that export controls didn't work, until I realized that China has leapfrogged the West on every technology that wasn't subject to export controls. And the only ones where they're still behind are the ones under export controls.
It’s far from self evident that without export controls they wouldn’t be significantly further behind since there would be less incentives to develop GPUs/etc. locally.
Why would US introduce export controls in sectors where they are already not very competitive?
It's insane how many articles have dropped in the past 24 hours that all have a similar tone. At this point it's hard to see this as anything other than a broad coordination from someone who wants to destroy Jensen/Nvidia.
There is a 1T IPO coming up. I'd be surprised if they didn't hand over plenty of bucks to PR companies, grass root campaigns, influencers and journalists.
More like that's how modern propaganda works. If it was about journalistic investigation, media outlets would not copy this story verbatim, but instead would check facts independently.
Not coordination necessarily. Trump and Howard Lutnick and a few others as reporting has indicated basically sat on allegations by other people (such as Rubio, Bessent, senators, other internal people) and seemingly looked the other way as Nvidia did some possibly-smuggling. Obviously while Trump was feeling powerful that put a lid on it. But it was only after the midterms when it’s clear Trump and others lost control that the leaks come out. Who knows from where exactly, but a lot of people wanted the Chinese chip exports to be way more strict. It’s a common policy position, and must be galling for some that Lutnick especially basically performed catch and kill for any allegations. Plenty of reporting about Treasury vs Commerce departments for example.
if we're going paranoid mode, on 9th of November Trump said that "leader of one large company connected to artificial intelligence" was almost sent to prison by Ministry of Justice due to wrongdoings connected to Ai, and this threat caused immediate correction by that company
> from someone who wants to destroy Jensen/Nvidia.
More like take over?
Perhaps the US administration cannot outright nationalize nvidia because of their free market narrative, so how about they make it a national security thing.
Also, that "AI is so dangerous please ban open source" that model providers put out does not seem to work so might as well go after the hardware providers which are the only moat.
Yep, pattern matching all of the "against NVidia" articles out there makes it pretty obvious that they are attempting to manufacture outrage against Jensen/Nvidia. I suspect Jensen didn't play ball with the powers that be, and so they replaced him.
I guess fingers crossed & good luck to them? After making consumer hardware unaffordable due to their greed I guess that is a reasonable outcome & karma at work.
I definitely care about this much less than it seems I'm supposed to.
I think the frontier models are definitely in a healthy dynamic where it seems the near-but-not-quite performance of open-weight models are what is pushing the most innovation and price competition from OpenAI, Anthropic and co
Some people seem to think that American sanctions and export controls are some form of absolute justice, and that any violation is an affront to humanity.
If GPUs are going to be priced out of consumer reach anyway, I would rather see the GPUs used to create open weight models than just to make these AI companies richer.
Well the problem is you can't prove they are only being used to create open weight models. You can't prove they aren't also creating closed weight models with military applications beyond what they've publicly open sourced.
Why should I care that China is making some super powered hacking model? We already know they do state sponsored APTs. It's just motivation to care about security and red team your server code.
Their government is not really the type to violently take over other countries and impose their laws. Taiwan is a special case because they see it as a rebel state on their own land. The rest of us are probably better off with China being the top superpower than Trump's America.
> Their government is not really the type to violently take over other countries and impose their laws.
Taiwan would like a word. Taiwanese govt was the original government in China before CCP violently took over in the aftermath of WW2 with help from the Soviet Union.
China was also involved in destabilizing Vietnam by backing the communists and then after having a fall out with the communists there, proceeded to engage in border conflicts for almost two decades before normalizing relations.
China hasn't lifted a finger for Ukraine. It hasn't lifted a finger for Venezuela. It hasn't lifted a finger for Iran. Nobody knows what China's long term intentions are or how China will change after Xi Jinping's reign.
You don't surrender your culture's lead in tech willingly as it could be existential down the road.
To be fair the relationship between China and Taiwan has been reasonably peaceful and highly economically beneficial to both sides over the last several decades (aside from occasional sabre rattling)
> China was also involved in destabilizing Vietnam by backing the communists
That was mainly the USSR? But regardless it was > 50-60 years ago.
> China hasn't lifted a finger for Ukraine
Yes but why would they care? US/Europe rarely does anything about brutal conflict in Africa or Asia either unless they threaten their strategic interests.
> It hasn't lifted a finger for Venezuela. It hasn't lifted a finger for Iran
You mean by defending the governments of those countries from the US? Well if anything that just proves their point China prefers stability over keep control of countries which were partially in their “sphere of influence”
You say "original government". The majority of Chinese people would say "fringe group propped up by the US to destabilize China". The people chose overwhelmingly to kick those guys out, using violence where necessary. Kind of like Ukraine overthrew their previous president. Oh, people can only choose when the US likes the choice?
> The majority of Chinese people would say "fringe group propped up by the US to destabilize China"
The ROC that became Taiwan was established in 1912 after fall of Qing dynasty. The U.S. didn't have anything to do with its establishment. The U.S. provided some financial support to ROC in the 1930s against Imperial Japan's invasion. Meanwhile it was the Soviets who interfered more than anyone in China's civil war.
> We recover traces of the models’ initial values, and then search for the hardware configurations and initial configurations that reproduce them
From my (very cursory, naive, layman's) glance at the summary, they're identifying and tracing some characteristics that specific hardware imparts into a model's initialized weights. But then concluding that this indicates the models were also "trained on" that same hardware.
But is that a valid assumption? Is it possible that a model was initialized on one system configuration, while the bulk of the training was performed on another?
In fact, it would seem reasonable for the "okay, I'm setting up this new model on my workstation, prepping it for training next month" pipeline step to be far removed from the "okay, now deploy the [initialized, but not yet trained] model across our giant fleet of beefy training machines".
> The method identifies the hardware that generated the starting weights, not every chip that later helped train the model. When a training run creates its weights on the same GPUs that will train it, the fingerprint points to both. But a lab could generate the weights on one system and move them to another, or continue training an existing model on new hardware. The weights alone cannot rule out those possibilities.
I have run some large open models on neoclouds and the KYC has been minimal, even if the USG stops all GPU exports it will be extremely difficult to crack down on Chinese renting GPUs on Western servers.
There will be a black market of renting IDs to KYC and gain access to the chips, and then the hosts will monitor their servers for what is being trained, it will be a cat and mouse game.
We are headed for an increasingly unfree computation landscape and the politicians are going to make demands that will be hard to deliver.
yes. The USA hasn't force an embargo on China to other countries (e.g. by threatening to cut off Swift access). We aren't technically at war either, so its highly unlikely that the US would go that far.
What is surprising, I guess, is that the US allows GPUs to be exported at all to most countries. Since...as long as they are exported to a country that is friendly to china (which is basically most countries on the planet), China has access to GPUs on which to train with, making GPU purchasing restrictions mainly symbolic.
They dont need to threaten other countries, they can just pass a law saying that anyone anywhere in the world selling or renting gpus to chinese companies is committing a crime. Then they can sanction any company that does it anyway and that makes it a crime for any american company to sell gpus to that company. There have already been quite a few criminal cases around the export ban.
That would be unenforceable though knowing American arrogance they would definitely try. It’s like China trying to stop their rare earths from going to American uses by requiring all companies that import them to track their usage.
Black & grey market devices marked up many times cost buyers heavily, while still suppressing motivation to develop sovereignty.
If nvidia selling to China tolerated for the realpolitik and gets undone by incoming powers to create an appearance of change, is it really the right thing to do? Is it the smart thing to do?
It's an interesting thought experiment, though, we will probably never know.
Why on earth would that be suppressing motivation to develop sovereignty? The whole thing makes developing sovereignty an existential priority. When one critical part of your AI technology stack is subject to whims of your adversary, it's not really fully your AI technology stack. Today they turn the tap partially on, but only unofficially, tomorrow they'll shut it off. Time to buy your own tap.
I don't know the exact reason, but all empirical evidence shows that it does. E.g. the whole of Europe still using Microsoft because they can, even though bits and pieces get cut off (like the ICC). If Microsoft cut off the whole of Europe, then Microsoft wouldn't have power over them any more.
Microsoft's days in many EU countries' government departments are numbered regardless of what Microsoft does. The understanding is there: a critical dependence on a product coming from a now-unreliable ally is a problem. It will, however, take many years for this to actually happen, because it's much easier said than done.
Yup, I think the same applies to traffic of arms regulations.. Private firms sell to countries they are 'allowed' to, passing it between organisations in a chain until they get to the original customer they couldn't sell to directly.. Cuts are taken along the way, and people making money look the other way..
When Xi visited, Trump offered to sell him US made weapons.
What's the chance that these GPUs got to China without Trump getting a cut somehow?
Pretty much zero. Trump sat on knowledge of the affair for an entire year. That's extremely guilty behavior, he was either extorting Huang or covering g for him.
The whole thing is the most corrupt thing we have ever seen at the presidential level.
> The newest models we could read from Alibaba (Qwen), Zhipu (GLM), Tencent, Xiaomi and StepFun all read as Hopper-trained. Meituan says its LongCat-2.0 was trained entirely on “AI ASIC superpods,” and we found no NVIDIA fingerprint in the model weights it’s released.
Yeah makes sense. They are in transition. This article doesn't really tell us anything new (or interesting). There's nothing wrong w Chinese labs using GPUs that are already in the country. An export ban just means reduced import of new GPUs
For the things people are worried, it would be more dangerous when they didn't carry those fingerprints anymore - because it would mean they've caught up and switched to their own.
i dont know what is expected of export controls... they are never effective. they also find western chips in iranian drones etc. there is a huge gray area of resellers who resel to resellers etc. its pretty much uncontrollable....
During the Cold War, America used various shell companies to get valuable titanium out of Russia for building spy planes. If someone is willing to pay, smugglers will find a way.
> Simply banning things causes the price to go up. That alone is good enough to motivate the policy decision.
No twice.
First, banning them doesn't necessarily even make the price go up. If all you have to do to get the product is put someone else's name on the form, that raises the price by a number that rounds to zero. It probably costs your own companies more in bureaucracy and paperwork to try to enforce the pointless rules than it costs the other one to put a different name on the form.
Second, suppose it actually worked somehow. What happens then? The other country pays the higher price temporarily while throwing out some big government subsidies to develop a domestic version of the technology. Soon your own companies have a new foreign competitor in the world market which is receiving large government subsidies.
> That alone is good enough to motivate the policy decision.
This may work when its against an smaller. When deployed agianst a (near) peer - there usually isn't a price to high for a field of national security interest. China could get all the GPUs it wants / the US could get all the rare-earth magnets it wants over the objections of the other country.
> It can get all the GPUs it wants... at a higher price.
That's implicit in "there's no price too high". However higher the price is, the difference is still a rounding error when your budget is trillions of dollars.
I present this short-form documentary — or animated infographic, if you will — on how much governments are prudent in saving money in areas considered of vital national interests, denominated in GPUs instead of dollars: https://www.youtube.com/shorts/F6jMsMBVB34
Export controls inherently raise the cost of doing business with parties you don't like, the other party has to either invest more resources acquiring them, or make the decision to forego acquiring them. The grey market exists precisely because there's an opportunity to exploit this for profit. At the end of the day, when these effects of these export controls are discussed, they usually seem to revolve around export controls setting a specific program or effort back by a span of months, not in stopping a specific commodity from accessing another market whatsoever (excepting some hyper-specific components).
As Hezbollah discovered in 2024, covert supply-chain interdiction is very possible and I'd be surprised if the CIA didn't know what was going on, at least to some extent.
Weird detail: some of these results are based on initial patterns in weights surviving through training. But isn't muon, which is used for many of these models, well-known to be able to consistently erase patterns in weights, unlike previous optimizers [1]?
This doesn't really call the overall results of the report in the question (the strongest finding is the perfectly matching weights for dead-weight rows which weren't touched by an optimizer at all), but it does suggest that the explanation in the twitter thread[2] of how this works is way too simplistic. Are the patterns in question somehow "special" and not erasable even with muon?
If you could rent compute to what is essentially a competitor and monitor everything about their code and models in the process, since you're still in control of the hardware, why would a company or in this case a country not take that competitive advantage?
No, they distinguish AMD vs Nvidia based on rounding or some similar difference farther down in the stack, finding that the values that would differ based on chip vendor are all consistent with an Nvidia platform. Search for "slightly differently".
Weights start as random values generated from a seed. Pytorch uses however many cores are available on the architecture to write out the initial values in parallel, so each seed + architecture pair has a particular signature. If I understand Will's technique correctly he's basically made a rainbow table of initial weight states. Training alters the initial weights, but if you find one seed + architecture pair that vastly more-closely matches the trained weights, you can infer it must have been the source of the initial weights.
Distillation might shape the weights similarly in some way to the source model, but that would come from the signal, not the noise. I think it would be very surprising if the initial untrained weights were encoded with any fidelity into the outputs such that training another randomly-initialized set of weights (likely of completely different size and shape) on them could leave a signature of the source model's initial state, does that make sense?
Not seeing how that could happen since all the "distillation" attacks i've seen are just token level, not logprob level, and definitely not vector level. So, I don't see how that could be the case.
1. access to the very best frontier models for hard problems
2. good enough open models from China, the US (I like poolside.ai, but lots of good choices), and even other countries like France that make smaller investments in Digital Sovereignty.
If the ‘super rich fucks’ in the US who have been making risky AI bets succeed in convincing the US government to curtail the use of small efficient models for easier problems, that will just hurt our economy.
@dang curious, is this post getting heavily downvoted or flagged?
I get that the geopolitical angle is sensitive, especially with the backdrop of today's big news - but I thought this was a really well-researched technical deep dive, right up HN's alley. (TIL: you could fingerprint gpu hardware just from an llm weights!)
I need to double check PyTorch code, I am not sure how robust this fingerprinting is without PyTorch "bug". Parallel philox based PRNG is a well-researched area and I am surprised (even if it is not secure-PRNG) it can fingerprinting like this. But obviously there are more people who are more knowledgeable on this can give their perspectives here.
Cute trick but it's not exactly a secret, e.g. deepseek recently published their own inference engine -- and it only has kernels for SM100 (blackwell) and Huawei Ascend 950, and all its published performance results using blackwell.
No one found it remarkable a couple weeks ago, ... but now the fact that DS is using blackwell is apparently some super secret requiring clever RNG init stats???
It’s perfectly legal to use Nvidia GPUs to train Chinese models. It’s called cloud compute, maybe the author heard of it?
This seems like a Jensen hit piece to me, making it seem like shady Nvidia practices are responsible for Chinese models.
GPU pearl clutching is ensuring that the USA loses the market in the long term. Robotics = local models. Robots will outnumber humans in 30 years. We are making sure that China throws their full ability into catching up their fab technology, and they have logistical and ecosystem advantages that the USA and Europe can never hope to match, not to mention the power production.
We’d be much better off if they kept buying our GPUs and couldn’t justify the enormous investment to catch up their fabs.
Ah, that means the recent huge open-source AI advances, that we all benefit from, directly and indirectly, come from Jensen Huang's China boycott circumvention.
Which means, in future, we maybe cannot expect such large capability increases.
and Jensen's "cousin" -- a woman who looks strangely just like him -- happens to be CEO of AMD. makes one wonder exactly just how much nepotism was involved in this clearly coordinated circular financing and sanctions evasion operation.
US frontier AI models carry the fingerprint of data acquired from illegal sources like Anna Archive, libgenesis, etc. and, (un?)surprisingly, nobody cares.
Only Anthropic was stupid enough to let be caught red-handed but get away with some ridiculously low penalty.
"Specially configured AMD hardware, which is also restricted from export to China, is the only alternative we have identified; we have excluded it for DeepSeek-V4-Flash"
This sounds as if they are using something VERY fuzzy to "match" things, and if they are ALSO matching against completely different hardware which this makes it sounds like.... then I dont know if this means anything.
They match the pseudo-random generator from the NVIDIA library cuRAND.
AMD has a library that emulates it, for compatibility.
Their methodology is interesting, but it only proves access to a single Blackwell GPU, which would have been sufficient for initializing an LLM with random values, before the start of its training.
There is no evidence whatsoever about which GPUs were used for training, except the supposition that if they had access to 1 Blackwell, they might have had access to more of them. Such a supposition may be justified about the number of cockroaches in a kitchen, but less so about the components of a training cluster, which might be heterogeneous.
The LLM could have been initialized on the workstation of a developer, before being transferred to a server cluster, for training.
The "not physically located within China" framing is doing a lot of work here, and under the EAR it isn't the operative test. Jurisdiction follows the item: a US-origin GPU stays subject to the EAR wherever it is re-exported, and a foreign-made one is caught if it exceeds the de minimis US-content threshold or falls under the foreign direct product rule. So the question isn't where the box sits, it's who the re-export is going to and what the exporter knows about end use — "knowledge" in the regulation includes the conscious disregard / high probability standard, which is how a reseller's log of "customer declines to state workloads" becomes evidence rather than a shield.
Two other mechanics people miss: releasing controlled technology to a foreign national is itself a "deemed export", so staffing matters as much as shipping; and the control lists are keyed to ECCN, not to marketing names, so the classification question is per-part-number and per-performance-threshold, not per-product-line.
None of that makes enforcement easy across borders — the jurisdiction argument above is exactly why the extraterritorial reach fights keep landing in the same place: the US can reach the item and the US-person nexus far more reliably than it can reach a foreign reseller's intent.
We need deep investigations at the CFTC and SEC level on who took short positions and put options on NVDA. My bet is there are Singaporean propshops backed by the likes of Goldman and others who are rightsizing this market.
Chinese AI companies are free to use any Nvidia GPU they want as long as the GPU is not physically located within China. It is entirely consistent with US export controls for Chinese AI companies to remotely use GPUs located in Singapore or Malaysia.
“In facilities across the region, Megaspeed rents Nvidia chips to Chinese tech giants such as Alibaba, an arrangement that does not violate current US export control regulations.” See [1].
[1] https://www.atlanticcouncil.org/dispatches/beyond-the-spy-sc...
Literally my first thought. Noone can stop them from training in data centers that are located elsewhere. It's not like some military hardware that needs to be physically shipped to China for them to make use of it. In fact why would they bother with Chinese compute providers if those are limited because of GPU export controls? They knew this for a long time and probably have been knocking on the door of providers in other countries for a while. Heck, they could be training inside the US. They already distribute their open model weights from the US.
The US definitely has the power to stop most of it. It just looks like the executive branch doesn’t yet have the legal authority to do it, but it’s in an advanced legislative state.
How are you going to monitor who is logging into a system, or who a job is being run for? If you want that level of control the machines should not be exported at all.
Datacenters are very large and not very mobile and full of very expensive export controlled equipment and training runs are large expensive contracts. This is not exactly a difficult context to enforce KYC rules in
You want KYC for renting compute?
One could for example default-allow experiments without default-allow huge runs.
Renting one GPU for one day vs 20 000 GPUs for two months or however many/long it takes to train one of these models.
Same way they do for Iran: if they catch you providing services to someone in Iran, they send you to jail for life. And they monitor all financial transactions so they'll notice.
Yeah but if you provide any service to someone in Iran it’s illegal. Meanwhile there are plenty of legitimate services that can be provided to China. That’s the difficulty. You can’t just monitor financial transactions because you don’t what the transaction is for.
They can do that to Americans. Harder for the rest of the world. How is the US going to monitor CIPS payments? How will they jail someone who never goes to the US? Just how much global jurisdiction is the US claiming?
In practice, the key ways that the US exerts influence outside of its own borders: (1) Sanction the company, which means any other company that does business with that company can also be sanctioned. (2) If enough companies are evading sanctions in a single country or the govt is helping those companies, then threaten to cut the country from USD settlement with the Federal Reserve, e.g., no USD FX allowed. That pretty much screws most economies. The "long arm" of the US govt is very, very, very long!
To be clear: I am not saying any of this is fair (or legal); I am only explaining how it works.
i expect this will raise crypto prices through forced realignment?
[dead]
Except the US has been threatening this to everyone over more and more issues and over the long its provided a movement off transacting in US dollars
If not USD, what currency? There are only two possible, realistic alternatives: EUR and CNY. Still, the USD is still wildly dominant. Read more here: https://www.jpmorgan.com/insights/global-research/currencies...
How is the money going to end up in a US account via CIPS?
Why would it need to end up in a US account?
That's where Jensen Huang is
Does he own any data centres outside the US?
> How will they jail someone who never goes to the US?
The same way they "jail" UN-appointed judges that go against their masters' interests: de-platform them from all US-controlled payment and software platforms.
That is not jail.
if you are referring to people at the ICC, they are cut off from US services, but that is a long way from being in jail.
Can't travel, can't shop, can't have a cellphone or even a landline...
Yes, they can. They cannot buy things from Amazon or book through booking.com, as that French judge said, but they can do things that do not require American systems. Do you have any evidence they cannot have phones? What stops they shopping in person? Or booking directly with a hotel?
> He said: “We are going 30 years back in time. It’s like a time machine returning us to a pre-digital world,” he told France TV.
https://vinnews.com/2026/03/23/icc-judge-who-issued-netanyah...
Considering the fact that booking.com is Dutch/European that means that they can’t do business with any company that has a significant presence in the US. Presumably that would also affect major hotel chains?
If you would be so kind as to read your own source, it answers your questions
Sanctions only become effective if they are targeted well enough to ensure compliance. If you start making them too broad they will fail, as happened in Russia.
The same way that SAAS and IAAS (etc) providers do now for citizens of other countries under sanctions - IPs, Credit Cards
So another company in Asia is going to execute the batch and send the result back to China. And you can add as many indirection levels as you like. This is going to be impossible to track.
You'd just invert it. Make it so you have to be approved under export controls as a customer like they were gonna do for mythos (and like how mythos sort of still works) and make it an allowlist instead of a denylist
If that company is acting in defiance of sanctions, like any other company that acts as a middleman to circumvent sanctions, and it is caught, it will face severe consequences.
There's no gotcha here, companies the world over could just ignore any sanctions they wanted, and some try to, but they do so knowing they themselves are (potentially) breaking local and international laws.
You might have well asked "what if a company helps you by importing contraband instead of you doing it directly"
None of them are breaking local and international laws by doing so, unless a country's ruling says other countries laws supersede their own
One of the key reasons that a country like the USA can enforce sanctions is that it can tell other countries that if their companies/citizens/etc break the sanctions, then the USA will put restrictions on those countries ability to trade with the USA, which is why only a few countries will trade with Iran, North Korea, and Russia
At some point those countries will also tire of the USA trying to tell them what to do and will tell the USA politely to fuck off. This is basically what India did with US sanctions on Russia and Iran.
Given the USA has completely torched its reputation amongst allies in the space of a year, I would expect this to happen more often.
Exactly right. The term I use is "economic network effect".
Does anyone know why does the US allows Canada to trade with Cuba? That one always stood out to me as a bit odd. My guess: Canada is the largest trade partner with the US, so the US probably has more to lose by sanctioning Canada.
The US can certainly tell other countries things. And they can even pretend to listen. Or not, as in the case of India and China's response to oil sanctions.
> trade with Iran, North Korea, and Russia
No way it could do the same to China without some major repercussions.
Yes, that's called a treaty, or a trade agreement. Remember all the fuss about Investor/State Dispute Settlement clauses? Most countries have them with each other by now.
To add to this, if the provider suspects that a customer is facilitating sanction busting, or could reasonably be expected to have foreseen it, that's enough for a discussion with lawyers...
Lol what consequences? Such shell companies can appear by the thousands every hour. Even if you ban 2 or 3 it doesn't matter.
Funny, sanctions have successfully applied to countries such as Russia, Iran, etc for decades...
They do try workarounds, and when those are caught there are consequences
You can sit back down now.
Edit: Is there going to be more "laws can be broken so let's not bother" arguments, because that's lame and boring
> sanctions have successfully applied to countries such as Russia, Iran, etc for decades.
Largely because other countries are willing to go along with it, and to the extent they are willing to do so.
Even NATO countries import Russian gas. The UK rolled back sanctions on Russia to import diesel earlier this year. Most of the rest of the world did not impose effective sanctions and lots of people trade with them (including huge economies like China).
The rest of the world is to dependent on China to want to get involved. The US might succeed in the current administrations attempt to split the world into new cold war spheres of influence (because people are scared of China too) but its not going to be easy, it will be painful, and its not going to get many countries to completely agree to what the US wants.
China is not Russia or Iran it has the means to retaliate economically and politically if pushed to far. Even sanctioning Russia required great effort and huge amounts of self-sacrifice in Europe (are American consumers willing to tolerate that?).
> They do try workarounds, and when those are caught there are consequences
Lol. Yeh we can all see how well the sanctions worked on Russia and Iran. Thanks for proving my point!
Again with the "laws are broken so let's not have them" argument.
You're going to make it the provider's problem, unleash the full wrath of the US Government on one or two who don't comply pour encourager les autres, and then it self-regulates(ish).
> pour encourager les autres
I was wondering why you wrote this part in French instead of English. Turns out it is a phrase that is sometimes loaned into English. It comes from the (French) writings of Voltaire.
https://wordhistories.net/2016/10/22/pour-encourager-les-aut...
How so? They would need to forbid nvidia to sell to anybody but US customers
Or our allies that we control. But in terms of how - remember the mythos launch? Im absolutely not saying it's necessarily a good idea, I'm just that it's not that hard if we want to do it.
Mythos is a closed model from a US company trained and deployed in the US. The article says they have no idea where and how Chinese models were trained, only that they used GPUs that can't be sold to China. Apart from sanctioning anyone who does business with China (which would probably hurt the US more than anyone) they can't really do anything about that.
That's fine, they'll just sell to a no-name company with a Santa Clara mailing address.
And regulate all the owners of nvidia hardware. Including, somehow, those who aren't in the USA.
how would that work?
"definitely has the power" lmao
Oh, legal authority is what usually stops this bunch /s
Follow the money.
They're also free to use Nvidia GPUs physically located in China. However, customs and border enforcement will try to stop them from moving GPUs physically located outside China to physical locations inside China. However, if they can bypass that, they can use the GPU.
This is the key statement. Lol
Is there any international regulation that forces China to comply with arbitrary US export controls? If not I can't see why Chinese companies are not free to use any Nvidia GPU they can get their hands on period (How the US would then react towards China, the Chinese company and Nvidia is another question).
There's no gpu police. Americans can't literally be everywhere monitoring everything, and they don't even try to.
Pine Gap says hi.
No there isn't. If they have an Nvidia GPU, then they have it
I see that here also:
https://archive.ph/BiSrT
"A key player in getting Megaspeed off the ground was Alibaba, the Chinese e-commerce leader that has also been an aggressive player in AI, said people familiar with the negotiations at the time. The U.S. had begun imposing export controls on advanced chips in late 2022, preventing Nvidia from selling them directly to China over concerns they could boost China’s military and surveillance capabilities.
"Alibaba was looking for a partner that could help it access computing power from data centers stuffed with Nvidia chips located outside China, the people said."
https://www.bloomberg.com/news/articles/2026-11-13/banned-nv...
[flagged]
The author of this piece is a SWE-turned-influencer "China hawk" from OpenAI (i.e. the primary advocate of export bans along with Anthropic), and this isn't an academic paper despite the theming.
I have no idea whether this is legit or not (there's no link to any code), I'm mentioning this because all the comments here are assuming truth without verifying the contents.
This post is from Will Depue
The supposed “China Hawk” you mention, on google searching, seems to be David Perdue.
Is there a chance you are confusing similar yet different last names as the same person?
No, the names are correct. OpenAI has quite a few SWEs who spend a lot of time advocating for positions on social media.
I don't understand. What difficulties would these Chinese companies face if they held their training overseas?
The US could impose secondary sanctions to close the loophole, or pressure these countries behind closed doors to not allow these data center uses. Since the only goal of the export restrictions is to hamper Chinese AI efforts, that would be a logical step. If there is one thing that is certain then it's that the current US administration doesn't care at all about open trade and international trade agreements.
So that's what we will become after the models have made our skills obsolete: security state lobbyists.
The method seems sound and the findings correct. But I think many are missing that (as the research itself says) this doesn't prove any illegal behavior. It's perfectly legal for Chinese companies to train models on GPUs that they rent outside of China. It's also legal for companies to buy GPUs and install them in Malaysia with the intention of renting them to Chinese companies.
There isn't even an allegation that the Chinese companies lied about this. They simply didn't disclose which chips were used. In all cases where they did disclose the training chips, this research found the disclosures to be accurate.
The small print in TFA states the many limitations of their method, i.e. that it actually provides no evidence about which GPUs were really used for training, and much less about who was their owner and where they were located.
The method just strongly suggests that the initial values of the weights, before starting training, were generated with the NVIDIA cuRAND library on a Blackwell GPU (or on an AMD GPU, where the NVIDIA library is emulated for compatibility).
The training might have been done with thousands of GPUs of other types. It is plausible that the GPUs used for training or at least a part of them may have been of the same type with the one used for initializing the model, but it is not proven in any way.
A scenario that matches what is reported in the article, but in which there is only a single Blackwell GPU involved, is if the LLM was intialized on the workstation of some lead developer, who had the privilege to use the best GPU they had, and then it was transferred to a server cluster for training, where other kinds of GPUs were used.
Now that this has been published, it would be very simple in the future for any Chinese AI company to avoid this "fingerprinting", by just using a different PRNG, instead of cuRAND.
Yes it's simple to avoid this fingerprinting now that people know about it, but nobody knew about it before. I don't believe people are casually transferring snapshots between chip architectures without a really good reason, either. It's hard enough to coordinate these training runs without involving multiple chip architectures in a single run.
Does 'legal' an 'illegal' apply here? Surely, the USA legal system only apply in the USA?
'Legal' and 'illegal' apply to Nvidia or other US companies facilitating these activities. Perfectly legal for Nvidia to supply GPUs to be rented by Chinese companies, or for Oracle to build datacenters for Chinese companies to rent, as long as they are installed outside of China.
You'd think so but the USA is also the country that's most advanced at getting its legal system to apply outside of it. E.g. they shut down the International Criminal Court.
When did they shut it down?
There were cases being considered as recently as yesterday. https://www.icc-cpi.int/case-records
https://news.ycombinator.com/item?id=50479876
So not "shut down" but unsubstantiated reports that one country is in talks with the US about limiting its powers.
>USA is also the country that's most advanced at getting its legal system to apply outside of it
Yes it has nukes. However china also has nukes. That poses some legal limitation.
The USA didn't need nukes to get Venezuela.
Indeed, but the USA has been flexing it's muscles against small countries all over the world for decades. However, go back and study the Korean war and Vietnam war, where China and even Russia intervened, and the US lost their arses and was sent packing both times. My eldest uncle was an officer/pilot in the USAF involved in the Vietnam war, flew many bombing missions. What do we have to show, for both those conflicts ?
It needed them to avoid retaliation from the rest of the world.
Did they really "get" Venezuela ? What actually prevents them to start ignoring what US wants in a while ?
it's funny because it convinces me that export controls don't work so why do them
I used to think the same, that export controls didn't work, until I realized that China has leapfrogged the West on every technology that wasn't subject to export controls. And the only ones where they're still behind are the ones under export controls.
It’s far from self evident that without export controls they wouldn’t be significantly further behind since there would be less incentives to develop GPUs/etc. locally.
Why would US introduce export controls in sectors where they are already not very competitive?
The purpose of a system is what it does, not what it doesn't do.
It's like the war on drugs, just one more crackdown will fix everything this time.
Its more like the war on bugs. Just because you can't kill them all doesn't mean you shouldn't spray and lay pest control.
because they are bad
It's insane how many articles have dropped in the past 24 hours that all have a similar tone. At this point it's hard to see this as anything other than a broad coordination from someone who wants to destroy Jensen/Nvidia.
There is a 1T IPO coming up. I'd be surprised if they didn't hand over plenty of bucks to PR companies, grass root campaigns, influencers and journalists.
This is how the news works on a story of major importance. As details of corruption come out, they all get reported.
One bigger aspect of this is that it was all saved from being leaked until right after the election.
Somebody was playing partisan politics with national security.
One key question: did Trump recently sell NVDA? How much and when? He's known of this for more than a year.
More like that's how modern propaganda works. If it was about journalistic investigation, media outlets would not copy this story verbatim, but instead would check facts independently.
Not coordination necessarily. Trump and Howard Lutnick and a few others as reporting has indicated basically sat on allegations by other people (such as Rubio, Bessent, senators, other internal people) and seemingly looked the other way as Nvidia did some possibly-smuggling. Obviously while Trump was feeling powerful that put a lid on it. But it was only after the midterms when it’s clear Trump and others lost control that the leaks come out. Who knows from where exactly, but a lot of people wanted the Chinese chip exports to be way more strict. It’s a common policy position, and must be galling for some that Lutnick especially basically performed catch and kill for any allegations. Plenty of reporting about Treasury vs Commerce departments for example.
if we're going paranoid mode, on 9th of November Trump said that "leader of one large company connected to artificial intelligence" was almost sent to prison by Ministry of Justice due to wrongdoings connected to Ai, and this threat caused immediate correction by that company
> from someone who wants to destroy Jensen/Nvidia.
More like take over?
Perhaps the US administration cannot outright nationalize nvidia because of their free market narrative, so how about they make it a national security thing.
Also, that "AI is so dangerous please ban open source" that model providers put out does not seem to work so might as well go after the hardware providers which are the only moat.
Yep, pattern matching all of the "against NVidia" articles out there makes it pretty obvious that they are attempting to manufacture outrage against Jensen/Nvidia. I suspect Jensen didn't play ball with the powers that be, and so they replaced him.
I guess fingers crossed & good luck to them? After making consumer hardware unaffordable due to their greed I guess that is a reasonable outcome & karma at work.
the Deep State has entered the chat.
I definitely care about this much less than it seems I'm supposed to.
I think the frontier models are definitely in a healthy dynamic where it seems the near-but-not-quite performance of open-weight models are what is pushing the most innovation and price competition from OpenAI, Anthropic and co
Some people seem to think that American sanctions and export controls are some form of absolute justice, and that any violation is an affront to humanity.
If GPUs are going to be priced out of consumer reach anyway, I would rather see the GPUs used to create open weight models than just to make these AI companies richer.
Well the problem is you can't prove they are only being used to create open weight models. You can't prove they aren't also creating closed weight models with military applications beyond what they've publicly open sourced.
Why should I care that China is making some super powered hacking model? We already know they do state sponsored APTs. It's just motivation to care about security and red team your server code.
Their government is not really the type to violently take over other countries and impose their laws. Taiwan is a special case because they see it as a rebel state on their own land. The rest of us are probably better off with China being the top superpower than Trump's America.
> Their government is not really the type to violently take over other countries and impose their laws.
Taiwan would like a word. Taiwanese govt was the original government in China before CCP violently took over in the aftermath of WW2 with help from the Soviet Union.
China was also involved in destabilizing Vietnam by backing the communists and then after having a fall out with the communists there, proceeded to engage in border conflicts for almost two decades before normalizing relations.
China hasn't lifted a finger for Ukraine. It hasn't lifted a finger for Venezuela. It hasn't lifted a finger for Iran. Nobody knows what China's long term intentions are or how China will change after Xi Jinping's reign.
You don't surrender your culture's lead in tech willingly as it could be existential down the road.
> Taiwan would like a word
To be fair the relationship between China and Taiwan has been reasonably peaceful and highly economically beneficial to both sides over the last several decades (aside from occasional sabre rattling)
> China was also involved in destabilizing Vietnam by backing the communists
That was mainly the USSR? But regardless it was > 50-60 years ago.
> China hasn't lifted a finger for Ukraine
Yes but why would they care? US/Europe rarely does anything about brutal conflict in Africa or Asia either unless they threaten their strategic interests.
> It hasn't lifted a finger for Venezuela. It hasn't lifted a finger for Iran
You mean by defending the governments of those countries from the US? Well if anything that just proves their point China prefers stability over keep control of countries which were partially in their “sphere of influence”
You say "original government". The majority of Chinese people would say "fringe group propped up by the US to destabilize China". The people chose overwhelmingly to kick those guys out, using violence where necessary. Kind of like Ukraine overthrew their previous president. Oh, people can only choose when the US likes the choice?
> The majority of Chinese people would say "fringe group propped up by the US to destabilize China"
The ROC that became Taiwan was established in 1912 after fall of Qing dynasty. The U.S. didn't have anything to do with its establishment. The U.S. provided some financial support to ROC in the 1930s against Imperial Japan's invasion. Meanwhile it was the Soviets who interfered more than anyone in China's civil war.
> We recover traces of the models’ initial values, and then search for the hardware configurations and initial configurations that reproduce them
From my (very cursory, naive, layman's) glance at the summary, they're identifying and tracing some characteristics that specific hardware imparts into a model's initialized weights. But then concluding that this indicates the models were also "trained on" that same hardware.
But is that a valid assumption? Is it possible that a model was initialized on one system configuration, while the bulk of the training was performed on another?
In fact, it would seem reasonable for the "okay, I'm setting up this new model on my workstation, prepping it for training next month" pipeline step to be far removed from the "okay, now deploy the [initialized, but not yet trained] model across our giant fleet of beefy training machines".
That's in the article:
> The method identifies the hardware that generated the starting weights, not every chip that later helped train the model. When a training run creates its weights on the same GPUs that will train it, the fingerprint points to both. But a lab could generate the weights on one system and move them to another, or continue training an existing model on new hardware. The weights alone cannot rule out those possibilities.
I have run some large open models on neoclouds and the KYC has been minimal, even if the USG stops all GPU exports it will be extremely difficult to crack down on Chinese renting GPUs on Western servers.
There will be a black market of renting IDs to KYC and gain access to the chips, and then the hosts will monitor their servers for what is being trained, it will be a cat and mouse game.
We are headed for an increasingly unfree computation landscape and the politicians are going to make demands that will be hard to deliver.
Matrix multiplication is a human right
Keep your laws off my weights.
as the post mentions, the models could have been trained on 1. GPUs within China or 2. GPUs outside China for example datacenters in Singapore.
The 2nd option is perfectly legal and allowed by US government.
The 1st option is also perfectly legal and allowed by the US government. Only the export of GPUs is illegal, and only sometimes.
Is the U.S. government OK with this if the money to buy those GPUs and run inference on them comes from Chinese companies?
yes. The USA hasn't force an embargo on China to other countries (e.g. by threatening to cut off Swift access). We aren't technically at war either, so its highly unlikely that the US would go that far.
What is surprising, I guess, is that the US allows GPUs to be exported at all to most countries. Since...as long as they are exported to a country that is friendly to china (which is basically most countries on the planet), China has access to GPUs on which to train with, making GPU purchasing restrictions mainly symbolic.
They dont need to threaten other countries, they can just pass a law saying that anyone anywhere in the world selling or renting gpus to chinese companies is committing a crime. Then they can sanction any company that does it anyway and that makes it a crime for any american company to sell gpus to that company. There have already been quite a few criminal cases around the export ban.
That would be unenforceable though knowing American arrogance they would definitely try. It’s like China trying to stop their rare earths from going to American uses by requiring all companies that import them to track their usage.
Then china stops selling goods to USA for a couple of weeks until they realise there's more in life than GPUs.
Or the initial starting weights could have been generated by an export-controlled GPU outside of China and the training done in China.
Or the initial starting weights could have been generated by a non-export-controlled GPU emulating Nvidia.
This whole thing is a nothingburger designed to stir engagement.
What if the gov tolerated gpu exports on purpose?
Black & grey market devices marked up many times cost buyers heavily, while still suppressing motivation to develop sovereignty.
If nvidia selling to China tolerated for the realpolitik and gets undone by incoming powers to create an appearance of change, is it really the right thing to do? Is it the smart thing to do?
It's an interesting thought experiment, though, we will probably never know.
Why on earth would that be suppressing motivation to develop sovereignty? The whole thing makes developing sovereignty an existential priority. When one critical part of your AI technology stack is subject to whims of your adversary, it's not really fully your AI technology stack. Today they turn the tap partially on, but only unofficially, tomorrow they'll shut it off. Time to buy your own tap.
I don't know the exact reason, but all empirical evidence shows that it does. E.g. the whole of Europe still using Microsoft because they can, even though bits and pieces get cut off (like the ICC). If Microsoft cut off the whole of Europe, then Microsoft wouldn't have power over them any more.
Microsoft's days in many EU countries' government departments are numbered regardless of what Microsoft does. The understanding is there: a critical dependence on a product coming from a now-unreliable ally is a problem. It will, however, take many years for this to actually happen, because it's much easier said than done.
That's what they say. What they actually do betrays otherwise.
It would suppress it because investing in this stuff is a very large short/mid term cost for long term gain.
This, providing access to the gpus but at a large markup plays into that dynamic.
Entities being myopic shouldn't be surprising.
Does the Chinese government strike you as an organization that's strapped for cash or infected by short-term thinking?
Yeah actually, they have engaged in tons of short term thinking to their detriment.
Look at their demographics issue, as well as things like the real estate bubble.
Their economy is below what it was supposed to be by a real margin.
Post-Trained To Reinforce Western Values
> What if the gov tolerated gpu exports on purpose?
Given the track record of this administration, they were probably paid to tolerate said exports.
Indeed, Jensen 'gifted' $25m to the ballroom after Trump was briefed on the matter. Many sources, here's one: https://www.newsweek.com/trump-jensen-huang-spy-claim-nvidia...
::shocked pikachu::
Yup, I think the same applies to traffic of arms regulations.. Private firms sell to countries they are 'allowed' to, passing it between organisations in a chain until they get to the original customer they couldn't sell to directly.. Cuts are taken along the way, and people making money look the other way..
When Xi visited, Trump offered to sell him US made weapons.
What's the chance that these GPUs got to China without Trump getting a cut somehow?
Pretty much zero. Trump sat on knowledge of the affair for an entire year. That's extremely guilty behavior, he was either extorting Huang or covering g for him.
The whole thing is the most corrupt thing we have ever seen at the presidential level.
> The whole thing is the most corrupt thing we have ever seen at the presidential level.
Bigger than Watergate by far
... So far
> The newest models we could read from Alibaba (Qwen), Zhipu (GLM), Tencent, Xiaomi and StepFun all read as Hopper-trained. Meituan says its LongCat-2.0 was trained entirely on “AI ASIC superpods,” and we found no NVIDIA fingerprint in the model weights it’s released.
Yeah makes sense. They are in transition. This article doesn't really tell us anything new (or interesting). There's nothing wrong w Chinese labs using GPUs that are already in the country. An export ban just means reduced import of new GPUs
For the things people are worried, it would be more dangerous when they didn't carry those fingerprints anymore - because it would mean they've caught up and switched to their own.
i dont know what is expected of export controls... they are never effective. they also find western chips in iranian drones etc. there is a huge gray area of resellers who resel to resellers etc. its pretty much uncontrollable....
During the Cold War, America used various shell companies to get valuable titanium out of Russia for building spy planes. If someone is willing to pay, smugglers will find a way.
A lot safer than smuggling since re-exporters generally don't get arrested unless they travel to the US.
Simply banning things causes the price to go up. That alone is good enough to motivate the policy decision.
> Simply banning things causes the price to go up. That alone is good enough to motivate the policy decision.
No twice.
First, banning them doesn't necessarily even make the price go up. If all you have to do to get the product is put someone else's name on the form, that raises the price by a number that rounds to zero. It probably costs your own companies more in bureaucracy and paperwork to try to enforce the pointless rules than it costs the other one to put a different name on the form.
Second, suppose it actually worked somehow. What happens then? The other country pays the higher price temporarily while throwing out some big government subsidies to develop a domestic version of the technology. Soon your own companies have a new foreign competitor in the world market which is receiving large government subsidies.
Export controls are a failed policy.
> That alone is good enough to motivate the policy decision.
This may work when its against an smaller. When deployed agianst a (near) peer - there usually isn't a price to high for a field of national security interest. China could get all the GPUs it wants / the US could get all the rare-earth magnets it wants over the objections of the other country.
It can get all the GPUs it wants... at a higher price.
> It can get all the GPUs it wants... at a higher price.
That's implicit in "there's no price too high". However higher the price is, the difference is still a rounding error when your budget is trillions of dollars.
I present this short-form documentary — or animated infographic, if you will — on how much governments are prudent in saving money in areas considered of vital national interests, denominated in GPUs instead of dollars: https://www.youtube.com/shorts/F6jMsMBVB34
Export controls inherently raise the cost of doing business with parties you don't like, the other party has to either invest more resources acquiring them, or make the decision to forego acquiring them. The grey market exists precisely because there's an opportunity to exploit this for profit. At the end of the day, when these effects of these export controls are discussed, they usually seem to revolve around export controls setting a specific program or effort back by a span of months, not in stopping a specific commodity from accessing another market whatsoever (excepting some hyper-specific components).
> its pretty much uncontrollable
As Hezbollah discovered in 2024, covert supply-chain interdiction is very possible and I'd be surprised if the CIA didn't know what was going on, at least to some extent.
Would be nuts but unsurprising if the cia is selling subtlety broken gpus to china
Yes or indeed supplying cards to the Chinese market with extras...
Weird detail: some of these results are based on initial patterns in weights surviving through training. But isn't muon, which is used for many of these models, well-known to be able to consistently erase patterns in weights, unlike previous optimizers [1]?
This doesn't really call the overall results of the report in the question (the strongest finding is the perfectly matching weights for dead-weight rows which weren't touched by an optimizer at all), but it does suggest that the explanation in the twitter thread[2] of how this works is way too simplistic. Are the patterns in question somehow "special" and not erasable even with muon?
[1] https://docs.modula.systems/examples/weight-erasure/
[2] https://twitter.com/willdepue/status/2120992457630326841
You don't need to do any fingerprinting, kernels for those GPUs have been in open-sourced code from the Chinese labs. It's not like they're hiding it
If you could rent compute to what is essentially a competitor and monitor everything about their code and models in the process, since you're still in control of the hardware, why would a company or in this case a country not take that competitive advantage?
Title is deceptive; at most they've shown it carries the fingerprint of "a 148-CU GPU".
No, they distinguish AMD vs Nvidia based on rounding or some similar difference farther down in the stack, finding that the values that would differ based on chip vendor are all consistent with an Nvidia platform. Search for "slightly differently".
one important thing is hidden in the notes under appendix2, this is not even mentioned in the conclusion.
> We recovered no signature from DeepSeek-V4.1-Flash (after an exhaustive search),
DS v4.1 is the latest version, no signature found in this model.
Not too many years before the headline reads
"Western AI models using GPUs export controlled by China"
The decline of the USA and the West has been precipitous.
Not doubting that they have access to these.
But, could the fingerprints be due to distillation from Blackwell trained models rather than training?
Weights start as random values generated from a seed. Pytorch uses however many cores are available on the architecture to write out the initial values in parallel, so each seed + architecture pair has a particular signature. If I understand Will's technique correctly he's basically made a rainbow table of initial weight states. Training alters the initial weights, but if you find one seed + architecture pair that vastly more-closely matches the trained weights, you can infer it must have been the source of the initial weights.
Distillation might shape the weights similarly in some way to the source model, but that would come from the signal, not the noise. I think it would be very surprising if the initial untrained weights were encoded with any fidelity into the outputs such that training another randomly-initialized set of weights (likely of completely different size and shape) on them could leave a signature of the source model's initial state, does that make sense?
Not seeing how that could happen since all the "distillation" attacks i've seen are just token level, not logprob level, and definitely not vector level. So, I don't see how that could be the case.
Just further proof that export controls rarely restrict the ones they are designed to restrict
The purpose of a system is what it does, not what it doesn't do.
So the purpose of MS365 was to go down for 12+ hours?
This is all so stupid.
US businesses need two things:
1. access to the very best frontier models for hard problems
2. good enough open models from China, the US (I like poolside.ai, but lots of good choices), and even other countries like France that make smaller investments in Digital Sovereignty.
If the ‘super rich fucks’ in the US who have been making risky AI bets succeed in convincing the US government to curtail the use of small efficient models for easier problems, that will just hurt our economy.
Awesome job, most interesting analysis I've read these days.
@dang curious, is this post getting heavily downvoted or flagged?
I get that the geopolitical angle is sensitive, especially with the backdrop of today's big news - but I thought this was a really well-researched technical deep dive, right up HN's alley. (TIL: you could fingerprint gpu hardware just from an llm weights!)
I need to double check PyTorch code, I am not sure how robust this fingerprinting is without PyTorch "bug". Parallel philox based PRNG is a well-researched area and I am surprised (even if it is not secure-PRNG) it can fingerprinting like this. But obviously there are more people who are more knowledgeable on this can give their perspectives here.
Cute trick but it's not exactly a secret, e.g. deepseek recently published their own inference engine -- and it only has kernels for SM100 (blackwell) and Huawei Ascend 950, and all its published performance results using blackwell.
And these were completely openly disclosed as headline properties of the software: https://github.com/deepseek-ai/DeepInfer/tree/main/#performa...
No one found it remarkable a couple weeks ago, ... but now the fact that DS is using blackwell is apparently some super secret requiring clever RNG init stats???
This article states the obvious?
It’s perfectly legal to use Nvidia GPUs to train Chinese models. It’s called cloud compute, maybe the author heard of it?
This seems like a Jensen hit piece to me, making it seem like shady Nvidia practices are responsible for Chinese models.
GPU pearl clutching is ensuring that the USA loses the market in the long term. Robotics = local models. Robots will outnumber humans in 30 years. We are making sure that China throws their full ability into catching up their fab technology, and they have logistical and ecosystem advantages that the USA and Europe can never hope to match, not to mention the power production.
We’d be much better off if they kept buying our GPUs and couldn’t justify the enormous investment to catch up their fabs.
Ah, that means the recent huge open-source AI advances, that we all benefit from, directly and indirectly, come from Jensen Huang's China boycott circumvention.
Which means, in future, we maybe cannot expect such large capability increases.
Then reduce the price of GPUs and RAMs. So we can own them or shut up and gtfo.
Are (ex-)OpenAI/Anthropic employees leading the charge here to protect their portfolios?
So export controls are more like export friction
and Jensen's "cousin" -- a woman who looks strangely just like him -- happens to be CEO of AMD. makes one wonder exactly just how much nepotism was involved in this clearly coordinated circular financing and sanctions evasion operation.
US frontier AI models carry the fingerprint of data acquired from illegal sources like Anna Archive, libgenesis, etc. and, (un?)surprisingly, nobody cares.
Only Anthropic was stupid enough to let be caught red-handed but get away with some ridiculously low penalty.
WAIT! Are you saying the giant plagiarism machine was built with…stolen IP?!?!
Ladies and gentlemen, we are now officially in Huang-gate.
Such a great domain name by the way.
My BS-meter just went through the roof:
"Specially configured AMD hardware, which is also restricted from export to China, is the only alternative we have identified; we have excluded it for DeepSeek-V4-Flash"
This sounds as if they are using something VERY fuzzy to "match" things, and if they are ALSO matching against completely different hardware which this makes it sounds like.... then I dont know if this means anything.
They match the pseudo-random generator from the NVIDIA library cuRAND.
AMD has a library that emulates it, for compatibility.
Their methodology is interesting, but it only proves access to a single Blackwell GPU, which would have been sufficient for initializing an LLM with random values, before the start of its training.
There is no evidence whatsoever about which GPUs were used for training, except the supposition that if they had access to 1 Blackwell, they might have had access to more of them. Such a supposition may be justified about the number of cockroaches in a kitchen, but less so about the components of a training cluster, which might be heterogeneous.
The LLM could have been initialized on the workstation of a developer, before being transferred to a server cluster, for training.
No shit sherlock https://x.com/CulperResearch/status/2054623962122912041
clickbait written as research....
Land of the free*.
*conditions apply
oh no, anyways
Well, d'uh...
The "not physically located within China" framing is doing a lot of work here, and under the EAR it isn't the operative test. Jurisdiction follows the item: a US-origin GPU stays subject to the EAR wherever it is re-exported, and a foreign-made one is caught if it exceeds the de minimis US-content threshold or falls under the foreign direct product rule. So the question isn't where the box sits, it's who the re-export is going to and what the exporter knows about end use — "knowledge" in the regulation includes the conscious disregard / high probability standard, which is how a reseller's log of "customer declines to state workloads" becomes evidence rather than a shield.
Two other mechanics people miss: releasing controlled technology to a foreign national is itself a "deemed export", so staffing matters as much as shipping; and the control lists are keyed to ECCN, not to marketing names, so the classification question is per-part-number and per-performance-threshold, not per-product-line.
None of that makes enforcement easy across borders — the jurisdiction argument above is exactly why the extraterritorial reach fights keep landing in the same place: the US can reach the item and the US-person nexus far more reliably than it can reach a foreign reseller's intent.
[flagged]
[dead]
I guarantee you this isn't an "accident". Whatever is being sent over there is loaded with its own special spyware lol
[flagged]
We need deep investigations at the CFTC and SEC level on who took short positions and put options on NVDA. My bet is there are Singaporean propshops backed by the likes of Goldman and others who are rightsizing this market.
didn't the culper report come out in march or something like that?