Why is it a problem that the Chinese labs are just distilling down Anthropic’s models? Aren’t Anthropic’s models not just distilling down other people’s work?
Feels like Anthropic crying do as I say not as I do.
What Anthropic is doing requires way more resources than what the Chinese labs are doing. So their complaint is that they do 95% of the work and the Chinese labs do the last 5% and call it their own.
An argument can be made that Anthropic is also only doing the last 5% of the work (because the content they are training on was the other 95%) but that's a bit more philosophical.
Yeah, the whole thing seems like a human centipede of rug pulling. Probably the same as it's always been. Curating AI knowledge should be something that we put our best researchers towards, but realistically I think we wind up with 2-3 highly biased nationalistic models that are constantly copying off each other's notes.
Thanks, that’s the nature of any business. Founders see a way to take existing knowledge and expertise, combine it in some novel or interesting way, and produce a new product
But if you take it deeper, didn't most of those authors rely on the work of others? Most of human knowledge is small advancements of things we already knew. Often by reorganizing what we already knew.
Is that not what the foundation models are? A new reorganization of existing knowledge?
So then the problem is that Anthropic seems hypocritical when they knowingly insert themselves into this chain, and then complain about people down-chain from them.
To remedy the negative impressions (if they even care to do so) they should do 1 of 2 things:
1) stop complaining about it
2) stop distilling other people's work
SOTA models cost hundreds of millions to train. Did creating the contents of the text corpus they were trained on really cost an equivalent of 20x as much (~10 billions)? I honestly don’t know, but I could imagine it having been significantly less.
This isn’t meant as a moral argument, just musing about the relative cost comparison.
If you look at movies alone that would easily surpass 10s of billions. The cost of most books is probably more nebulous, but books, research, and more all have time and money spent to create them. I would guess the corpus of all media from the 20th century on would be minimally in the hundreds of billions of dollars.
I get that angle but it’s a weak argument as Anthropic is doing the same to others. Also while there’s certainly a lot of computing power needed to do what Anthropic does, it’s increasingly clear there isn’t much secret sauce involved. Everyone knows how do to the core work it’s just a question of who wants to burn billions on compute to do it.
Anthropic’s anger here seems mostly rooted in their annoyance that this exposes they don’t really have core IP that’s not just easily replicated. And that’s clearly a problem for a deeply unprofitable company trying to convince people they’re worth $2 trillion.
> What Anthropic is doing requires way more resources than what the Chinese labs are doing.
Oh that’s very sad.
Meanwhile Anthropic made a product from the work effort of millions of people without compensating them, sell that product on tap and unless I am mistaken do not even have their competitors’ cover of having released an open weights model.
They have taken from culture (including very specifically their most direct customers’ specific culture), turned it into a machine to make themselves rich, and then want to dump themselves onto pensions funds and ordinary savers to carry the bag.
It is, I agree, philosophical, because karma is a philosophy.
And the communal work of humanity is orders of magnitude more work than what anthropic pays for their scraping of content. I got no check from them for my contributions
Indeed, if it isn't a crime to train on humanity's data, it isn't a crime to train on capitalism arranged frontier LLM provider models. Is that bad for shareholders and capitalism? Meh, sounds like a suboptimal socioeconomic systems issue. Burn up all the capital the unsophisticated are willing to provide. “We are selling to willing buyers at the current fair market price.”
With my apologies to Brewster Kahle, "Universal Access to All Knowledge." [2]
But that’s not more philosophical. It’s a perfect parallel! Enormous amounts of work, vacuumed up and resold. What’s the difference? If it’s ok to vacuum up all the knowledge in the world, then that includes knowledge of how to use all that to power an LLM.
It is kind of ironic that they scraped the web for publicly available data and used it freely to train their models and now their freely available models are being used to train other models.
I'm overall pro-Anthropic and pro-banning open-weights AI, but I agree with the parent commenter; distilling Claude models is not that different from pretraining on web data. It's all basically the same sort of thing.
I think a good litmus test here would be if Anthropic were to not care about distilling their models when the distillers keep the resulting models closed-source and sell tokens via an API. If they cared only about security concerns and not about people profiting off of their work, then they should be publicly fine with this and only protest against it going into open-weights models.
From one perspective, the 5% estimation is near-infinite orders of magnitude off, since they've trained on something approaching the sum-total of human knowledge.
The root comment was asking how it is a problem. It one considers it wrong, then it’s a problem regardless of whether Anthropic is complaining or not. Anthropic’s complaining or non-complaining should have no bearing on whether it’s considered a problem or not.
Because no one outside the AI scientists understand what distilling means. They probably all think about Mash and a vodka still, and a completely unrelated association.
It ain't gonna happen. At work I have a dropdown menu in vscode with a dozen models to use interchangeably. They're all essentially commodities and will compete on price and squash almost all profit margin.
That's not their business model. They won't win on price, but they won't compete on price. Their business model is making the current state of the art.
If I'm a business and I need something done today, and bc Anthropic has the best model, there's a 99.9 chance it will be completed successfully for $1000. And using Deepseek there's a 70% chance it will, for $10 - you or me will go for the $10. Big businesses don't. Bc 1000 per task is nothing to them.
The business I work for is absolutely sensitive to 10 vs 1000, depending on the task. And it's a multi-billion dollar business. 1000/task may not be much on it's own, but there are a lot of tasks.
Actually opposite occurs. Big businesses are ok with a mediocre but cheaper result. Very few are willing to pay such cost. Just look at tech wages and the distributions
Big business doesn't pay more for better, but the do pay more for predictability, support and targeted outcomes. They will happily trade a chance at 100% better results for 10% less chance of unplanned outcomes
Except businesses are going the opposite direction here. The lack of stickiness makes the “premium” argument hard to play. Oracle won because swapping databases is a giant PITA. Swapping models requires almost no effort for most uses. And because of that enterprises are all building model marketplaces where providers have to compete on price performance.
Most folks I know can choose from any of the big labs or open weight models and they get billed internally for tokens against their budget. There’s little incentive to no switch to the lower cost closers.
This setup is a nightmare scenario for the big labs trying to execute the traditional enterprise sales plays. Those only work if your product is sticky and AI models are one of the least sticky things in the history of tech.
Because cost of original training >> cost of distilling. It's the same thing that happens with Chinese knockoffs of physical products - it takes a lot of money and R&D time to design a new product, but it's basically free to buy the product, reverse engineer it, and resell it. All the data they originally trained on was available for free on the internet. If the original work was so valuable, it shouldn't be up on the internet for free in the first place imo.
I'm also grateful to the Chinese labs for providing workarounds for the walled gardens that the US based AI companies are attempting to create.
Does anyone know if there are any distillation datasets available? I'd love to see these distributed on BitTorrent. I think it's critical that AI be democratized and not isolated in the hands of a few private companies.
As opposed to what? The US? You think the US is any different? Literally our pedo president publicly admits to insider trading. You think the US government gives a rat's ass about the American people?
It is in the interest of everybody-except-OpenAI and Anthropic that those companies have capable competitors; better still for the competitors to share their advances.
They don't have to be our friends to act in our interest.
I agree it would be very cool if they were open sourced, but the article is talking about the KV cache technique which if I understand correctly is open source (or at least the white paper is published).
It's crazy how articles like this get spawn-camped by people like this trying to throw this zinger in. Both AI conpanies in the US and the chinese companies with ccp desks in the corner can be bad. The good path forward is locally hosted AI models, that's known.
It's VERY clear that the US companies are trying to push for regulation to kill open models and open weights. I see this as much more hostile and authoritarian response than what we're seeing come out of China right now.
So is China going to always publish in the open? No clue. But right now they're modeling much better behavior.
You ask about distillation but I wonder, is there any training datasets (~ TB-order) available that startup folks in SV use or is it so that everyone has to create their own scraping pipeline ?
There’s a libertarian sentiment that that doesn’t sit well with “AI is harming people” sentiment. If AI has harmful uses, and I think anyone sensible would have to agree that it does, then giving everyone unrestricted AI is likely to make it worse.
It’s sort of like gun nuts arguing that more guns is the answer. I mean, ok, maybe you’re a responsible gun owner or AI user but relying on personal responsibility doesn’t fix systemic problems. There are bad people out there.
If food has harmful uses, and I think any one sensible would have to agree that it does, then giving everyone unrestricted food is likely to make it worse.
You can do this with cars, tools, computers, ... whatever you want. So, no, I think your point is wrong.
Safety laws are the wrong category, the equivalent regulations would be those that forbid the use of cars for drive-by shootings or as robbery getaway vehicles, and regulations against the use of food to provide crime energy.
> Haven’t tried getting a gun in California. How bad is it? How could it be improved?
You have to pass a basic knowledge test, have a clean background, prove residency in the state, be 21 (or 18 for hunting rifles, IIRC) and then wait 10 days.
It’s not game over because AI doomers are wrong in their projections. LLMs are a transformative technology like the internet, but they’re also overhyped and the useful applications aren’t as broad as people think they are. They’re also not Skynet.
I didn't say I don't trust anyone. Don't put words in my mouth, it's a sign of bad faith.
Other countries have governments that have earned that level of trust. I believe the US could get there eventually, but it will take a very long time because it has a very long way to go.
Gun owner enters the conversation. In high trust societies, armed people are very polite people. I don’t want the bad guys out there being the only ones with guns. Besides I like to shoot just like you like (whatever you like to do that is legal).
Replace “gun” with anything and you will see how your comment falls apart.
What’s next? A registry for food purchases? Your beer gut is starting to show.
If you mean high-trust societies like maybe Switzerland, I agree that it can work, but the US has lots of guns and doesn’t seem very polite, so how we’re doing it doesn’t seem to be working very well.
We do have lots of food safety regulation, which has more to do with selling food.
All concerns balance against competing concerns, and in this case freedom of computing and knowledge wins over safety. Especially since it’s trivial to copy and share open models.
Okay, you’re asserting that but I disagree. Why should anyone else be convinced? Why can’t we get the good uses without the harms? It doesn’t seem like an unavoidable tradeoff.
Well then do your best, people like me will keep sharing models just fine. (Not like I even do anything with them, but I’m a compulsive data hoarder.) It’s not like the copyright industry has been able to stop sharing either, and there’s serious money at stake there.
I predict that the AI scaremongering will fizzle out when the bubble bursts. There will still be die-hard believers but the public will lose interest.
I don’t see how a stock market crash will make AI-related concerns go away. There was a dot-com crash but the Internet just kept getting bigger and causing more problems. In many ways security has improved, but we worry more than ever about social media, etc.
I would say that there are very few people on earth that I trust less than Sam Altman, Dario Amodei and Elon Musk. Also my own government claims to have used Anthropic models to bomb a girls' school in Iran. If you combine US regulations with sociopathic private companies, you get into a worst case scenario for humanity imho. Again, I'm thankful to China or any other entity pushing open models, local models and even distribution of this technology.
The best explanation is that it's a goal of the CCP to generally commodotize LLMs, because LLMs will ultimately be a compliment to manufacturing (which China dominates), and you always want to "commodotize your compliments".
I think this explains why they are open sourcing broadly. It's not to be nice. It's a strategic play by the Chinese government to help ensure there are many players in this race and not too much power accumulates to American labs (even if American labs benefit in the process)
Totally agreed. People are so ready to praise China for their free models, but they aren't doing it because they believe in free open-source software. If China ever gets ahead, they're going closed source and weights immediately.
I'm beyond thankful that Chinese AI models are so good. I desperately want us to cure the myriad of maladies that humans suffer needlessly with on a daily basis. We're going to need more powerful models than we have now if we're gonna do that and the Chinese are providing the competition needed to push this thing as fast as we can.
I realize "going as fast as we can" is not the most popular position atm. But I'm far more interested in what good we can do than 10% apocalypse scenarios. I volunteer with a charity for childhood brain cancer and I do not want to see another 4 year old die. I'm willing to risk anything to stop this.
This really is a place where the guns-don't-kill-people argument applies, even moreso than guns themselves. The U.S. government massacred school children in Iran. Why does it matter how they targetted them?
Do you mean that you don't believe in the 10% apocalypse scenarios or that you think they're an acceptable risk? Only the latter is really "risk anything".
I've been running Qwen 3.8 27b (an opus 4.6 tier model), locally on a 5090 for just over two weeks @ 170 tokens/s. That's a frontier model from 9 months ago running on consumer hardware. Who knows where distillation and pruning gets us in another year.
Weird, I kinda gave up on 3.8 as anything other than a planner. I had it try to write some basic unit tests for an admittedly complex bit of code and it ran out of context thinking about the problem and exploring random parts of the code base repeatedly before it even wrote a single line. Toning down the thinking helped some, but then it wasn't much better than qwen coder.
In all fairness there are probably a lot of folks who picked one up for around MSRP (even if one of the board partner cards with an MSRP 10-15% over the FE).
Local inference will have a boom of cheap, powerful, and available cards at some point (even if it isn’t until 2028/2029). At some point the hyperscalers, and frontier labs, will face the capex problems that everyone talks about, and NVidia, AMD, Apple, and Intel will want to keep selling products.
Powerful, by today’s standard, local inference needs to be accessible to really unlock the “AI” economy long term. It’s just like how the move from mainframes to the PC 40ish years ago unlocked the “computer revolution.”
$9k is a small price to pay to experience the rapturous glory of AGI. I'd easily pay up to 3 times that to comfortably run the superintelligent models released in this post RSI world.
Indistinguishable or very mildly better. But it's considerably faster. Some portion of that is also probably down to improvements in model harnesses, I've been using opencode.
“So the Chinese labs have thrown a lifeline to the Western loss-making labs, and I just have no clue as to why.”
I don’t think it is intentional but this is actually quite bad for the western labs.
The entire booster narrative has been “look at how their revenue is growing! $10bn to $100bn ARR in under a year! This’ll be a multi-trillion IPO!” and the extrapolated future growth from $100bn to $500bn and $500bn to $1tn justified future investment… but that revenue was just because inference was expensive.
The revenue growth story is all that matters pre-IPO. If revenue falls from $100bn to $50bn that’s very very bad optics for OpenAI and Anthropic even if they are now profitable, it completely destroys the growth narrative.
I don't see why revenue has to fall even if marginal costs drop off a cliff. As long as they have the best models (perceived or otherwise) and can make security and IP guarantees that satisfy enterprise, and no firm with similar guarantees undercuts them on price (why would they want a race to the bottom?) they can have high revenue and high margin.
unless the major players collude they don’t get decide if they are in a race to the bottom. The best model was compelling 6 months ago when everyone was too impressed to care about price but that has worn off now and clients are paying attention to price. The best model is no longer a license to charge any amount.
It is funny that so many comments vascilate between "It is so expensive these companies can't make money and will go bankrupt in seconds" and "Inference is so cheap that these companies can't make money and will go bankrupt in seconds".
I never take them seriously, I just assume they are coming from countries that don't understand how capitalism works or are operating out of bad faith. The underlying reality of the market is always changing and needs are always changing. Some AI companies will fail, that is a given. Remember alta-vista? Yahoo? Did search go away? How about Microsoft phones? Nokia? Motorola?
OpenAI and Anthropic are not in the inference business. That is a commodity. They need to sell products and solutions.
They’re not contradictory positions. Inference is too expensive now to make money because the industry is immature and hasn’t yet optimized for financial success while customers don’t care much about price because they’re more concerned about not missing out.
Inference will be too cheap long term to make money because it is being commoditized and customers will start to care about results and not just be wowed by impressive technology.
And of course this technology will continue to exist but that is irrelevant to the business. OpenAI investors don’t care if LLMs exist in 10 years, they care if their investment in OpenAI has made money.
A random accusation of western lab using DeepSeek caching technique being top of HackerNews, this article feels more awkward than the AI race for me. It's a shame for all HackerNews viewers.
Right now you are right -- even if I ran my local LLM all day, the quality is not nearly as great, and it runs slowly -- so I use the tier one AI subscription services as they are faster and smarter. But that might only be true for a limited amount of time, and a limited number of circumstances.
To borrow your steam engine analogy, if local LLMs get as good as a Toyota Prius, even if OpenAI / Anthropic offer Ferraris, most people will be happy with their Prius as their daily driver.
Similarly, if the big labs start raising prices or cutting usage, you won't be able to use it as much as you want -- whereas a local LLM will run all day every day without costing you any extra money.
So right now you are right, but who knows how long that will last.
I find this too. In fact, recently I've been pushing more and more to the latest and greatest model every time there is an update. It just saves me so much headache.
The margins on NVidia datacenter hardware are ... high. At least one order of magnitude larger than a consumer chip.
Given the recent deepseekv4.1 advances - how good of a 3B model can we make to run on an iphone natively? is it good enough to match common muse/dot use cases for consumers? the phone is already always on.. no need for a cloud server.
> So the Chinese labs have thrown a lifeline to the Western loss-making labs, and I just have no clue as to why.
"Thus the expert in battle moves the enemy, and is not moved by him."
They figured out a clever method for avoiding excessive training costs via distillation. That forces the hand of frontier labs to move faster, produce better models, etc. (to avoid embarrassment and 'falling behind'—all the while shouldering most of the cost), which they can just keep distilling—or applying other techniques against—much to the dismay of said frontier labs.
There are 1000 Chinese labs. They are involuting, they cannot coordinate and the state won't let them coordinate because the state wants domination, not actual profits for anybody. So the market forces are allowed to dominate.
Because of the basic huge recession going on in China, you can't actually make money in China doing China things. So they gotta gird up their export stuff and try to export. That entails strong relations with American companies, American PR, English stuff, etc.
If you want an essay about this from a VC, read this one
What a strange way to put it. Market forces are a good thing in a market economy. Only someone who secretly wants or hopes for monopolies would you say something to the contrary (a VC).
The party has done something about enormous involution in solar panels, for example (https://www.csis.org/analysis/chinas-solar-industry-upheaval...) and previously steel. They're planning something for cars. They don't on LLM because of the newness and the wish for preeminence.
Pouring 100% of your tech optimism, and probably portfolio, into a product that some say is about to get Wile E Coyote flattened by market dynamics would probably inspire serious market skepticism.
China has a different perspective on it, they believe that there is such a thing as harmful competition and they are willing to step in to stop it.
Enshittification and related problems can be a result of market forces just as much as they can be a result of monopoly/duopoly or a small cartel. Excess competition sometimes results in all firms scraping the barrel to squeeze out pennies, especially with technology (such as large online marketplaces) making pricing more transparent.
Marx actually predicted that ever-intensifying competition would destroy markets through overproduction, although he did not use the term involution.
Price wars are good even if they lead to more bad products on the market. If quality is important people will pay more for it and ignore the bad ones, if not then everybody saves some money.
The reason it doesn't work like that IRL is centralized marketplaces. If the winning strategy on Alibaba is low prices, bad quality and botted reviews to compensate then every seller has to do it to survive, because they can't get buyers outside the platform. That's not excess competition. It's a lack of competition just on a different level.
I'd call it overcapacity instead. A lot of investment without capital discipline making sure there is actually return of investment leading to too much supply.
Not that OpenAI, Anthropic or SpaceX aren't doing the same.
My immediate midwit take is: doesn't matter if it helps Anthropic/OpenAI if it helps DeepSeek more, relatively. Making open-weights models even cheaper and easier to run expands that "market" and increases competitive pressure on the Big Two, who still have to charge money.
Idk, but it's doing a lot for my view of China. Maybe that's a point? Maybe they just want their own innovation to go as fast as possible and they don't care that other countries also benefit? A rising tide lifts all boats? They are already known for the best manufacturing, they're just adding software dev to the list? Maybe they just want to undermine the US in a non-aggressive way?
Why did we (the west) ever start open sourcing anything? Maybe we just like sharing? Maybe humanity only grows on pre-competitive layers like Linux and clean water. Maybe, the chinese government is closer to their people, and does not let large companies influence them and just doesn't like closed private hyperscalers with a lot of power?
(Some points assume the government has a role in the openness, which I think is likely)
I think it’s fairly simple: they’re forced into this situation by being late and worse in terms of capabilities. They’re not far behind, but as long as they’re behind they’ve needed to give people some reason to try and use their models. Cost is one factor. But it probably wasn’t enough. Being open has given them a lot of attention. Free marketing. Good will.
Put another way: if they were not cheaper and open, they would simply not be competitive. They would already be dead.
I don’t think this ends well for the Chinese labs. This is going pretty much like I thought. Western labs is just copying their improvements (I don’t think publishing the techniques matter here.. they’d just hire to gain the knowledge or figure it out themselves), and they have access to more GPUs and have better branding, so in the end where can the Chinese labs compete? Even lower cost? Open weights? I’m not sure open is a sustainable way to compete either. Eventually there will be some fully open source AI models that cuts out that avenue of competition as well.
This looks like a speed run of the history of analytics DBMS’s.
Once upon a time, everyone had a secret sauce in network or data encoding or query optimization, but in the last ~10 years computational physics and economics have basically decided the “correct” architecture and everyone (including OSS) has converged.
Yeah, would have been nice if the author put a bit more thought into this article to come up with something rather than to just give up once they've reached the point of their article lol
It's just their usual national strategy like what they did to solar pane and EV. The solar panel industry is mostly dominated by China and their profit rate is basically ... negative. The EV industry in China is in similar condition, where the average profit rate is only 1.5%. Their upstream suppliers are also hold as hostages that most of them won't get their money back within 6 months.
The weird ideology here is to dominate the market at ANY COST, even it benefits the opponents.
The post makes an assumption that US labs did not already possess similar optimization. It's also very possible that they did, but are simply not telling anyone in order to maintain obscene margins on cached read, similar to AWS' absurd pricing on bandwidth.
Yes - the Chinese labs serve a market that rely on open-weight models and managed deployments, and the labs gain competitive relevance by releasing those models. The cache optimization feature they came up with required new software to utilize on the inference-end, meaning that open source software would need to be specifically updated to work with these models. It wasn't the type of advancement that they could even theoretically keep secret.
The chinese don't view AI as metaphysical, they view it as an engineering challenge they consider good for their state and want to dominate the global market like they do with batteries/EVs/photovoltaics. They want to proliferate them as much as possible and traditionally they don't care much for IP. They also want hardware makers to make optimized chips specifically for these models.
My guess would be that if they "help" western labs becoming better, then any break-throughs they (western labs) make after that, is also a benefit to the Chinese labs - if they can distill the models.
Basically, western labs are in it for the money / commercial monopoly. Chinese labs are in it for the tech? As long as they can keep distilling models, and get access to research other ways, they benefit. And if they can push western labs forward, they'll benefit from that themselves.
All the comments here are very US/Western-centric. Maybe they are a different culture, having a completely different economic model. Maybe they are not Capitalists and not thinking in pure Capitalistic terms, such as winning, growth, market domination, IPO, market value or competition. Maybe they are not obsessed with US labs. Maybe they are ideologically different than you. Maybe they have different priorities. Maybe they have a different playbook. Maybe they never thought of it as throwing a lifeline to American labs. Maybe they don't care. Maybe they're just different people.
Because it's entirely possible that Western labs already did this optimization but didn't publish it, and then the Chinese figured it out and decided to brag about it.
We don't know either way, so I find the whole thing silly to speculate on.
Unpopular, maybe, but what about the normal reasons? The researchers are looking to make a name for themselves, and/or they genuinely care about AI advancement.
AI fundamentally insecure. Vulnerable to forcing hallucinations via search results. Vulnerable to invocation of commands in data stream. More AI means more vulnerabilities.
I can't even fathom the trend these days of "we don't review the code" from security team perspective.
They get to make US labs look dumb and provide an open alternative that anyone can host themselves.
They're building bridges over the moats that companies with far too much US investment are trying to build, and if they do it continually it can help destabilize the US economy.
Open releases are just the obvious move when you're not the incumbent. You commoditize the thing your competitors charge for and get distribution you could never buy.
> So the Chinese labs have thrown a lifeline to the Western loss-making labs, and I just have no clue as to why.
Because contrarily to the author's assumption, all labs, Western or not, have sufficient skills to discover the optimizations anyway, and publishing or not is not actually that important?
The doublethink required to simultaneously believe "our safeguards prevent our models from doing unsanctioned cybersecurity tasks" and "distillation is why Chinese models are getting better at cybersecurity tasks" is genuinely quite funny.
How co-designed are these optimizations with the model itself? I'd imagine you can't just stick post-training adapters onto existing architectures for these things, or am I wrong?
I really want to explore the inference space, but it seems like many of the inference optimizations are coming from model-hardware codesign. I don't seem to recall many generic "inference engine" optimizations since prefill/decode disagg a year ago.
This matters for me since I want to break in but the bar seems to be understanding the actual theory of the training process now too given the codesign happening, and I'm not the richest guy on the block lol
Watch any interview with the Chinese AI leaders and compare it to the unending doomer word salads from America's mightiest paper billionaires. We're losing because we have a loser late stage capitalist scarcity mindset. They're winning because they're sharing notes and one-upping each other just like we used to until 2015 or so. They have a healthy ecosystem of competing small AI startups. We have two bloated unprofitable pigs both striving to be too big to fail. My money's on China for the immediate future.
To answer the author’s question of why Chinese labs give away their work for less than cost, the answer is involution. China is struggling with overcompetition in other areas of its economy as well, such as electric cars, and perhaps ironically its labor share of income is substantially lower than the U.S.
it's funny to me how the success/not total implosion of htese companies is predicated on profitable, revolutionary-tier success, and that it's increasingly possible that the profits will never really materialize. pretty interesting move on China's part.
Yeah, and Anthropic probably got inspired to this new fast read-in thing for making agentic stuff make more sense from the latest DeepSeek model. Maybe it was in the pipeline, but it clearly has the same effect and DeepSeek had published it by the point Anthropic dropped their prices for reading tokens in, so they may well have copied it.
> If you read the news headlines these days, you would be forgiven for thinking that the Western labs are getting spawn-camped by Chinese labs en masse.
As far as I can tell, neither of the frontier US labs have referred to distillation as "stealing", but someone please provide a link if I'm wrong.
They do claim that it violates their ToS, which we can assume is simply correct, since they get to put whatever they want in their ToS.
Given all that, I don't know what the fuss is. Are they supposed to not use the advances that were openly published by Chinese labs? The entire industry is built on a discovery made at Google, which was published openly. Should Chinese labs therefore not use transformers? Should US labs not try to prevent distillation of their models?
I don’t think there is fuss, just the author sharing the information and mentioning how they find it a bit ironic that US labs expenses can be reduced drastically thanks to the Chinese companies they continuously frame as adversaries
It's actually a little funny that this whole thing is predicated on "beating" the Chinese, when (as they have been for the past 4 decades, and the Japanese before them) they're perfectly happy to let us do the bulk of the work and then swoop in with a svelte, cheap, user-friendly version right after. One part Apple, one part Dollar Store.
Is the thinking that the day or so between US systems achieving ASI and Chinese systems doing the same, we'll figure out a way to neutralize them indefinitely? Because otherwise, none of this makes much sense. And it only starts to swerve back to sanity if the assumption is that this isn't a race or competition, but instead a joint effort to achieve something good for humanity. But you can't really delta profit off that, can you?
>The new game in town is adopting Chinese labs’ advances. Note how I call this adoption instead of the more vitriol-infused “stealing” that Anthropic tends to use.
I mean, there's a pretty big difference between labs publishing their research openly and a competitor utilizing it versus a lab breaking TOS to... hmmm, what's the word? steal data from a competitor?
>That’s because, unlike the Western companies, the Chinese are pretty much giving away their recipes.
Yeah, Western AI companies have never published their research. It's crazy how the Chinese had to independently develop the foundational technology that powers LLMs because Western companies simply never publish their research (I mean, as long as you ignore stuff like this <https://arxiv.org/abs/1706.03762>).
>The latest one shamelessly copied without acknowledgement is the breakthrough in KV cache optimizations that DeepSeek has generously shared with the world.
Thank you, generous corporation. I'm sorry that other corporations don't provide you free publicity for your selfless contributions to the world.
>Now I don’t know why they would freely give away such a breakthrough, but they just did
Well I'm glad the author finally got to their point. A very insightful analysis.
>They do seem to be a little embarrassed by the copying. Hence the silent releases without much pre-announcement for both Claude Opus 5.5 and GPT-6.1 Sol.
You have to be in pretty deep to infer this kind of emotion to these kinds of corporate activities.
>So the Chinese labs have thrown a lifeline to the Western loss-making labs, and I just have no clue as to why.
Then why write this article? Why point out these things just to have no conclusion?
This article sucks. Even if you hate US AI labs and are all aboard Chinese labs producing open models, there's nothing of substance here. This is the loose draft that you hand to your LLM to finish for you, but it seems the author just forgot to do so.
Even if you're willing to characterize US AI labs as evil and selfish and Chinese AI labs as righteous and generous (which is already completely trivializing these dynamics to the extent that anybody over the age of 14 can likely identify is lacking nuance), you can at least put some effort into producing some hypotheses about why these dynamics are occurring. Of course, odds are if the author did try to articulate some hypothesis, they'd likely quickly realize that the narrative they're painting just doesn't hold up.
Color me skeptical that OpenAI and Anthropic’s researchers had never thought to dig into these optimizations, and instead are just spending hundreds of billions on data centers and custom ASICs.
This is an extremely thin analysis that has obviously been voted to the top of the homepage because HN hates the big labs.
Deepseek's innovations are published as research. There's nothing 'assumed' about this. The common slur repeated in the West that Chinese labs are parasitic distillers is totally absurd when so many genuinely valuable advances and contributions to the field are published openly by China's labs.
I am talking about correlating Anthropic/OpenAI cache prices going down with Deepseek publication - neither of those labs have said that's what they used for example.
And the only data they are showing is that cache prices went down for new Claude/OpenAI models but that's proving nothing, IMO.
I’m incredibly skeptical that OpenAI is spinning up custom ASICs for improved inference performance, but they never thought of optimizing KV cache until a tiny Chinese lab did it? Give me a break.
Why is it a problem that the Chinese labs are just distilling down Anthropic’s models? Aren’t Anthropic’s models not just distilling down other people’s work?
Feels like Anthropic crying do as I say not as I do.
What Anthropic is doing requires way more resources than what the Chinese labs are doing. So their complaint is that they do 95% of the work and the Chinese labs do the last 5% and call it their own.
An argument can be made that Anthropic is also only doing the last 5% of the work (because the content they are training on was the other 95%) but that's a bit more philosophical.
But the analogy still holds.
The original authors of all the text, creators of the media and developers of the software did far more work than Anthropic.
Yeah, the whole thing seems like a human centipede of rug pulling. Probably the same as it's always been. Curating AI knowledge should be something that we put our best researchers towards, but realistically I think we wind up with 2-3 highly biased nationalistic models that are constantly copying off each other's notes.
Thanks, that’s the nature of any business. Founders see a way to take existing knowledge and expertise, combine it in some novel or interesting way, and produce a new product
But if you take it deeper, didn't most of those authors rely on the work of others? Most of human knowledge is small advancements of things we already knew. Often by reorganizing what we already knew.
Is that not what the foundation models are? A new reorganization of existing knowledge?
Yes this is all true.
So then the problem is that Anthropic seems hypocritical when they knowingly insert themselves into this chain, and then complain about people down-chain from them.
To remedy the negative impressions (if they even care to do so) they should do 1 of 2 things: 1) stop complaining about it 2) stop distilling other people's work
It’s a few corporations stealing work from others, to sell it back to us. That’s it.
not at that scale though
quite the leap from “authors rely on the work of others” to vacuuming up the sum total of digitized knowledge to tune some matrices
Don't forget the mothers of all those original authors, as well as everyone who labored to build and sustain the societies which produced writers.
It all comes full circle with chinese models being available free for all humanity to use
SOTA models cost hundreds of millions to train. Did creating the contents of the text corpus they were trained on really cost an equivalent of 20x as much (~10 billions)? I honestly don’t know, but I could imagine it having been significantly less.
This isn’t meant as a moral argument, just musing about the relative cost comparison.
If you look at movies alone that would easily surpass 10s of billions. The cost of most books is probably more nebulous, but books, research, and more all have time and money spent to create them. I would guess the corpus of all media from the 20th century on would be minimally in the hundreds of billions of dollars.
I asked Claude to estimate the cumulative salaries of US only journalists over the last hundred years:
$500B for all kinds including TV and online
$300B for newsrooms including all staff
$140B for newsroom reporters only
So yeah, I think the price of the information ingested is way higher than training costs
I suspect so. It is a lot of data. You're looking at essentially all publicly available (and some non public) intellectual work.
Given they ingested basically the whole internet and then some, you cannot possibly be serious when you mean it's worth less than 10 billions.
The totality of the content on internet is worth several orders of magnitude more.
Simple math:
A training set of 15 trillion tokens is 10 trillion words.
A penny a word is cheaper than the cheapest beginner freelance writer.
That makes a training set of 10 trillion words cost $100B.
Lots of assumptions there for sure, but we're certainly in the ballpark you are describing.
Yes, Easily, and by multiple orders of magnitude.
Yes, duh! Human output across the millennia is worth much more than whatever is being invested in frontier labs.
How is this even a question.
Kinda agree statistical modeling relies on all the hard work, effort, passion and risk taken from just about everybody.
All the text and so on had unrealized potential. Without Anthropic et al it would remain unrealized.
It’s like FTL. Until someone realizes it, it’s just talk.
He didn’t say it was an analogy. He said both are distillation.
I get that angle but it’s a weak argument as Anthropic is doing the same to others. Also while there’s certainly a lot of computing power needed to do what Anthropic does, it’s increasingly clear there isn’t much secret sauce involved. Everyone knows how do to the core work it’s just a question of who wants to burn billions on compute to do it.
Anthropic’s anger here seems mostly rooted in their annoyance that this exposes they don’t really have core IP that’s not just easily replicated. And that’s clearly a problem for a deeply unprofitable company trying to convince people they’re worth $2 trillion.
> What Anthropic is doing requires way more resources than what the Chinese labs are doing.
Oh that’s very sad.
Meanwhile Anthropic made a product from the work effort of millions of people without compensating them, sell that product on tap and unless I am mistaken do not even have their competitors’ cover of having released an open weights model.
They have taken from culture (including very specifically their most direct customers’ specific culture), turned it into a machine to make themselves rich, and then want to dump themselves onto pensions funds and ordinary savers to carry the bag.
It is, I agree, philosophical, because karma is a philosophy.
And the communal work of humanity is orders of magnitude more work than what anthropic pays for their scraping of content. I got no check from them for my contributions
Indeed, if it isn't a crime to train on humanity's data, it isn't a crime to train on capitalism arranged frontier LLM provider models. Is that bad for shareholders and capitalism? Meh, sounds like a suboptimal socioeconomic systems issue. Burn up all the capital the unsophisticated are willing to provide. “We are selling to willing buyers at the current fair market price.”
With my apologies to Brewster Kahle, "Universal Access to All Knowledge." [2]
[1] https://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior
[2] https://www.youtube.com/watch?v=RV_ALlJGU_c
This feels like a straw man argument. The parent comment didn't say it was a crime
Isn't it better for planet? By not doing the wasteful transformation work again
Absolutely. I'm not taking a side here, I'm just pointing out why Anthropic might have a valid complaint.
That complaint is invalidated by the argument of tu quoque. Complaining about something they are doing themselves.
> but that's a bit more philosophical
It sounds exactly the same, not more philosophical to me, except one is more inconvenient.
Wait, wasn’t 95% of the work creating the content in the first place?
Can you elaborate on why the second is “a bit more philosophical”?
I see absolutely no distinction between the two, aside from minor technical approaches to gathering the content.
But that’s not more philosophical. It’s a perfect parallel! Enormous amounts of work, vacuumed up and resold. What’s the difference? If it’s ok to vacuum up all the knowledge in the world, then that includes knowledge of how to use all that to power an LLM.
It is kind of ironic that they scraped the web for publicly available data and used it freely to train their models and now their freely available models are being used to train other models.
I'm overall pro-Anthropic and pro-banning open-weights AI, but I agree with the parent commenter; distilling Claude models is not that different from pretraining on web data. It's all basically the same sort of thing.
I think a good litmus test here would be if Anthropic were to not care about distilling their models when the distillers keep the resulting models closed-source and sell tokens via an API. If they cared only about security concerns and not about people profiting off of their work, then they should be publicly fine with this and only protest against it going into open-weights models.
How does Anthropic’s boot taste?
They spend 95% of the money, perhaps, but burning compute is not the same as doing the work.
I'm calling "work" here the conversion of energy to LLMs.
> What Anthropic is doing requires way more resources than what the Chinese labs are doing.
And writing a book requires many more resources than what anthropic does
I mean, 95% of the work if you don't factor the work to create the training data in the first place...
also, the actual work is the _copyrighted material created by the world_.
You are going to be surprised to hear how many resources were necessary to create all the data Anthropic is digesting
From one perspective, the 5% estimation is near-infinite orders of magnitude off, since they've trained on something approaching the sum-total of human knowledge.
“We’re both thieves, Steve. We both stole from xerox. You’re just mad I got their first.”
Two wrongs don’t make a right. (If you consider them as wrongs.)
It's not that the Chinese companies are right, it's that Anthropic has no place to complain about stealing.
The root comment was asking how it is a problem. It one considers it wrong, then it’s a problem regardless of whether Anthropic is complaining or not. Anthropic’s complaining or non-complaining should have no bearing on whether it’s considered a problem or not.
"That’s called competition. You’re allowed to test somebody else’s products all you want." - Jensen Huang https://x.com/wallstengine/status/2104604118937735553
Because no one outside the AI scientists understand what distilling means. They probably all think about Mash and a vodka still, and a completely unrelated association.
The word itself is the pivot, not anything else.
> They probably all think about Mash and a vodka still, and a completely unrelated association.
I don't know anyone with even a passing understanding of how LLM training works that thinks that is the appropriate analogy.
cool, do you think these media representations are for you, or 99% of the people who would love China to be sanctioned because they're foreigners?
“Distilling” is a funny way to say “learning from”
$$$$
it's not complex. there's hundreds of billions of investor dollars counting on vendor lock in and walled gardens
It ain't gonna happen. At work I have a dropdown menu in vscode with a dozen models to use interchangeably. They're all essentially commodities and will compete on price and squash almost all profit margin.
That's not their business model. They won't win on price, but they won't compete on price. Their business model is making the current state of the art.
If I'm a business and I need something done today, and bc Anthropic has the best model, there's a 99.9 chance it will be completed successfully for $1000. And using Deepseek there's a 70% chance it will, for $10 - you or me will go for the $10. Big businesses don't. Bc 1000 per task is nothing to them.
The business I work for is absolutely sensitive to 10 vs 1000, depending on the task. And it's a multi-billion dollar business. 1000/task may not be much on it's own, but there are a lot of tasks.
Also, is it really 99.9% vs 70%, or 99.9% vs 99%?
Yes, large corporations frequently pay orders of magnitude more for slightly better software. That's why Oracle produces the best stuff on the planet.
The real issue is that Deepseek has a 99.7% chance. So I can run it 10 times until it works and still pay 1/10 the money.
Actually opposite occurs. Big businesses are ok with a mediocre but cheaper result. Very few are willing to pay such cost. Just look at tech wages and the distributions
Big business doesn't pay more for better, but the do pay more for predictability, support and targeted outcomes. They will happily trade a chance at 100% better results for 10% less chance of unplanned outcomes
That 70% chance of success goes to 99.9% in 6 repetitions.
Big businesses might pay $1000 vs $60 for certain tasks, but that won't work out well at scale.
Except businesses are going the opposite direction here. The lack of stickiness makes the “premium” argument hard to play. Oracle won because swapping databases is a giant PITA. Swapping models requires almost no effort for most uses. And because of that enterprises are all building model marketplaces where providers have to compete on price performance.
Most folks I know can choose from any of the big labs or open weight models and they get billed internally for tokens against their budget. There’s little incentive to no switch to the lower cost closers.
This setup is a nightmare scenario for the big labs trying to execute the traditional enterprise sales plays. Those only work if your product is sticky and AI models are one of the least sticky things in the history of tech.
Sorry but it’s looking more and more like the top American labs won’t have any kind of moat.
>Aren’t Anthropic’s models not just distilling down other people’s work?
Can you elaborate on that? I mean my direct answer would be no, of course not. But why do you think frontier models are distilled?
Because cost of original training >> cost of distilling. It's the same thing that happens with Chinese knockoffs of physical products - it takes a lot of money and R&D time to design a new product, but it's basically free to buy the product, reverse engineer it, and resell it. All the data they originally trained on was available for free on the internet. If the original work was so valuable, it shouldn't be up on the internet for free in the first place imo.
I'm also grateful to the Chinese labs for providing workarounds for the walled gardens that the US based AI companies are attempting to create.
Does anyone know if there are any distillation datasets available? I'd love to see these distributed on BitTorrent. I think it's critical that AI be democratized and not isolated in the hands of a few private companies.
An authoritarian regime is not your friend and will pullback the moment their own models become highly capable.
Oh no, they might stop doing the thing that benefits me and that they were never required to do in the first place.
It can be predatory pricing that will bite us later.
It really can't be when the models are open weights and there's zero lock-in for inference providers.
As opposed to what? The US? You think the US is any different? Literally our pedo president publicly admits to insider trading. You think the US government gives a rat's ass about the American people?
Agreed, the US is not your friend and will pull the rug from under you at the first convenient opportunity.
It is in the interest of everybody-except-OpenAI and Anthropic that those companies have capable competitors; better still for the competitors to share their advances.
They don't have to be our friends to act in our interest.
There's no "pulling back" things that have already been open sourced.
Unfortunately most of the Chinese models are open weights and not open sourced.
I agree it would be very cool if they were open sourced, but the article is talking about the KV cache technique which if I understand correctly is open source (or at least the white paper is published).
Intelligence wants to be free
Yes, we already discussed the US.
It's crazy how articles like this get spawn-camped by people like this trying to throw this zinger in. Both AI conpanies in the US and the chinese companies with ccp desks in the corner can be bad. The good path forward is locally hosted AI models, that's known.
Right, which is why I'm happy to see China continue to innovate in the open, and increasingly wary of the US stance given articles like
https://www.anthropic.com/research/glm-5-3-and-the-spread-of...
It's VERY clear that the US companies are trying to push for regulation to kill open models and open weights. I see this as much more hostile and authoritarian response than what we're seeing come out of China right now.
So is China going to always publish in the open? No clue. But right now they're modeling much better behavior.
Just like how Anthropic and OpenAI is already doing?
You can be grateful to individual things an enemy does.
It's already happening now. They just banned engineers of their top AI labs and their families from leaving the country.
China is much more authoritarian than the US, but at this point it's like comparing two types of metastatic cancer.
The point is get what you can from both to develop open models, data and tools.
While your friend is Sam Altman, or US megacorps ?
Or did they not pull back when their models allegedly became highly capable, with the whole mythos debacle ?
Chill, Buddy. Why are you so anti-American?
This is equally true for US AI
>distillation datasets
You ask about distillation but I wonder, is there any training datasets (~ TB-order) available that startup folks in SV use or is it so that everyone has to create their own scraping pipeline ?
There are several? And there exists companies whose entire business is just providing them? iirc
Given the amount of people complaining about crawlers in the past 2 years, I think it's the latter
It's particularly important we all scan and destroy our own unique books!
there's https://pirateface.co/ which is like huggingface but distributed via torrents
The Chinese models are to an extent that distillation data.
There’s a libertarian sentiment that that doesn’t sit well with “AI is harming people” sentiment. If AI has harmful uses, and I think anyone sensible would have to agree that it does, then giving everyone unrestricted AI is likely to make it worse.
It’s sort of like gun nuts arguing that more guns is the answer. I mean, ok, maybe you’re a responsible gun owner or AI user but relying on personal responsibility doesn’t fix systemic problems. There are bad people out there.
If food has harmful uses, and I think any one sensible would have to agree that it does, then giving everyone unrestricted food is likely to make it worse.
You can do this with cars, tools, computers, ... whatever you want. So, no, I think your point is wrong.
We do in fact have car and food safety laws. Regulation is normal.
Safety laws are the wrong category, the equivalent regulations would be those that forbid the use of cars for drive-by shootings or as robbery getaway vehicles, and regulations against the use of food to provide crime energy.
Guns are highly regulated. Try getting a gun in liberal California, where Reagan screwed us.
Now, what I want to regulate are accordions.
I am a responsible accordion owner and I think we’re doing ok :)
Haven’t tried getting a gun in California. How bad is it? How could it be improved?
> Haven’t tried getting a gun in California. How bad is it? How could it be improved?
You have to pass a basic knowledge test, have a clean background, prove residency in the state, be 21 (or 18 for hunting rifles, IIRC) and then wait 10 days.
Nice try Philipp Mainländer!
I simply don't trust the people who would decide who gets AI (or guns) to make good choices.
This is a common populist sentiment, but if you don’t trust anyone then nothing can be done. Is it just game over?
It’s not game over because AI doomers are wrong in their projections. LLMs are a transformative technology like the internet, but they’re also overhyped and the useful applications aren’t as broad as people think they are. They’re also not Skynet.
Certainly no “anyone” and who happens to be worse than no one.
I didn't say I don't trust anyone. Don't put words in my mouth, it's a sign of bad faith.
Other countries have governments that have earned that level of trust. I believe the US could get there eventually, but it will take a very long time because it has a very long way to go.
Okay, sorry about that. But how do we start? Maybe there are AI safety organizations that deserve our support?
This is a common populist argument
Gun owner enters the conversation. In high trust societies, armed people are very polite people. I don’t want the bad guys out there being the only ones with guns. Besides I like to shoot just like you like (whatever you like to do that is legal).
Replace “gun” with anything and you will see how your comment falls apart.
What’s next? A registry for food purchases? Your beer gut is starting to show.
If you mean high-trust societies like maybe Switzerland, I agree that it can work, but the US has lots of guns and doesn’t seem very polite, so how we’re doing it doesn’t seem to be working very well.
We do have lots of food safety regulation, which has more to do with selling food.
All concerns balance against competing concerns, and in this case freedom of computing and knowledge wins over safety. Especially since it’s trivial to copy and share open models.
Okay, you’re asserting that but I disagree. Why should anyone else be convinced? Why can’t we get the good uses without the harms? It doesn’t seem like an unavoidable tradeoff.
Well then do your best, people like me will keep sharing models just fine. (Not like I even do anything with them, but I’m a compulsive data hoarder.) It’s not like the copyright industry has been able to stop sharing either, and there’s serious money at stake there.
I predict that the AI scaremongering will fizzle out when the bubble bursts. There will still be die-hard believers but the public will lose interest.
You’re not releasing new models though.
I don’t see how a stock market crash will make AI-related concerns go away. There was a dot-com crash but the Internet just kept getting bigger and causing more problems. In many ways security has improved, but we worry more than ever about social media, etc.
I agree that it isn't an unavoidable tradeoff in principle, but looking back at our (as in humanity) track record, it is 99% likely to be.
Populist doomers will tell you that nobody can be trusted, no global problems will ever be solved, nothing can be done, game over, don’t even try.
People did use global agreements and regulation to fix the ozone hole, though, so I think there’s a chance.
Actually I think profits win over safety.
> giving everyone unrestricted AI is likely to make it worse
Worse for whom?
The only effective defense against predatory corporate and government AI is personal protective AI.
Anything else is unilateral disarmament. It's the only way individuals can survive in the worse case scenario.
> gun nuts
Guns are different. They can't protect you against the government, contrary to gun nut claims.
I would say that there are very few people on earth that I trust less than Sam Altman, Dario Amodei and Elon Musk. Also my own government claims to have used Anthropic models to bomb a girls' school in Iran. If you combine US regulations with sociopathic private companies, you get into a worst case scenario for humanity imho. Again, I'm thankful to China or any other entity pushing open models, local models and even distribution of this technology.
Maybe there are other organizations that deserve our support?
The best explanation is that it's a goal of the CCP to generally commodotize LLMs, because LLMs will ultimately be a compliment to manufacturing (which China dominates), and you always want to "commodotize your compliments".
I think this explains why they are open sourcing broadly. It's not to be nice. It's a strategic play by the Chinese government to help ensure there are many players in this race and not too much power accumulates to American labs (even if American labs benefit in the process)
*complement
Commoditizing one’s compliments is a different strategy.
Totally agreed. People are so ready to praise China for their free models, but they aren't doing it because they believe in free open-source software. If China ever gets ahead, they're going closed source and weights immediately.
It's also a huge propaganda opportunity to influence the distribution of groupthink.
I'm beyond thankful that Chinese AI models are so good. I desperately want us to cure the myriad of maladies that humans suffer needlessly with on a daily basis. We're going to need more powerful models than we have now if we're gonna do that and the Chinese are providing the competition needed to push this thing as fast as we can.
I realize "going as fast as we can" is not the most popular position atm. But I'm far more interested in what good we can do than 10% apocalypse scenarios. I volunteer with a charity for childhood brain cancer and I do not want to see another 4 year old die. I'm willing to risk anything to stop this.
But you’re ok with AI being used to kill school children in Iran?
This really is a place where the guns-don't-kill-people argument applies, even moreso than guns themselves. The U.S. government massacred school children in Iran. Why does it matter how they targetted them?
i don't know much about medicine so i'm curious about how you're using AI, and how medicine in general is using AI
Do you mean that you don't believe in the 10% apocalypse scenarios or that you think they're an acceptable risk? Only the latter is really "risk anything".
I've been running Qwen 3.8 27b (an opus 4.6 tier model), locally on a 5090 for just over two weeks @ 170 tokens/s. That's a frontier model from 9 months ago running on consumer hardware. Who knows where distillation and pruning gets us in another year.
Weird, I kinda gave up on 3.8 as anything other than a planner. I had it try to write some basic unit tests for an admittedly complex bit of code and it ran out of context thinking about the problem and exploring random parts of the code base repeatedly before it even wrote a single line. Toning down the thinking helped some, but then it wasn't much better than qwen coder.
$9k for a 5090 now? Sheesh.
Well, I have a $750 card that runs at about 50-60% of that token rate :)
which one?
In all fairness there are probably a lot of folks who picked one up for around MSRP (even if one of the board partner cards with an MSRP 10-15% over the FE).
Local inference will have a boom of cheap, powerful, and available cards at some point (even if it isn’t until 2028/2029). At some point the hyperscalers, and frontier labs, will face the capex problems that everyone talks about, and NVidia, AMD, Apple, and Intel will want to keep selling products.
Powerful, by today’s standard, local inference needs to be accessible to really unlock the “AI” economy long term. It’s just like how the move from mainframes to the PC 40ish years ago unlocked the “computer revolution.”
$9k is a small price to pay to experience the rapturous glory of AGI. I'd easily pay up to 3 times that to comfortably run the superintelligent models released in this post RSI world.
Except you can do that cheaper by renting compute
What are you thoughts on it's performance compared to 4.6?
Indistinguishable or very mildly better. But it's considerably faster. Some portion of that is also probably down to improvements in model harnesses, I've been using opencode.
Well how’s it been so far
Frontier from 9 months ago? I don’t know about that. But it sure punches above its weight.
“So the Chinese labs have thrown a lifeline to the Western loss-making labs, and I just have no clue as to why.”
I don’t think it is intentional but this is actually quite bad for the western labs.
The entire booster narrative has been “look at how their revenue is growing! $10bn to $100bn ARR in under a year! This’ll be a multi-trillion IPO!” and the extrapolated future growth from $100bn to $500bn and $500bn to $1tn justified future investment… but that revenue was just because inference was expensive.
The revenue growth story is all that matters pre-IPO. If revenue falls from $100bn to $50bn that’s very very bad optics for OpenAI and Anthropic even if they are now profitable, it completely destroys the growth narrative.
I don't see why revenue has to fall even if marginal costs drop off a cliff. As long as they have the best models (perceived or otherwise) and can make security and IP guarantees that satisfy enterprise, and no firm with similar guarantees undercuts them on price (why would they want a race to the bottom?) they can have high revenue and high margin.
unless the major players collude they don’t get decide if they are in a race to the bottom. The best model was compelling 6 months ago when everyone was too impressed to care about price but that has worn off now and clients are paying attention to price. The best model is no longer a license to charge any amount.
Costs dropping opens you up to competition on price.
There isnt a lot of money in enterprise ai. Also my enterprise company gives me glm.
It is funny that so many comments vascilate between "It is so expensive these companies can't make money and will go bankrupt in seconds" and "Inference is so cheap that these companies can't make money and will go bankrupt in seconds".
I never take them seriously, I just assume they are coming from countries that don't understand how capitalism works or are operating out of bad faith. The underlying reality of the market is always changing and needs are always changing. Some AI companies will fail, that is a given. Remember alta-vista? Yahoo? Did search go away? How about Microsoft phones? Nokia? Motorola?
OpenAI and Anthropic are not in the inference business. That is a commodity. They need to sell products and solutions.
They’re not contradictory positions. Inference is too expensive now to make money because the industry is immature and hasn’t yet optimized for financial success while customers don’t care much about price because they’re more concerned about not missing out.
Inference will be too cheap long term to make money because it is being commoditized and customers will start to care about results and not just be wowed by impressive technology.
And of course this technology will continue to exist but that is irrelevant to the business. OpenAI investors don’t care if LLMs exist in 10 years, they care if their investment in OpenAI has made money.
A random accusation of western lab using DeepSeek caching technique being top of HackerNews, this article feels more awkward than the AI race for me. It's a shame for all HackerNews viewers.
Performance optimizations don't just help the western labs, they also help with running more powerful/useful LLMs locally.
If local LLMs get "good" enough, people will soon paying for subscriptions to ChatGPT and Claude, which hurts their revenue.
For nearly all tasks, I want the fastest and smartest AI model.
It is vanishingly rare I ask an older model to do any task. Newer bigger and smarter models will just do the task better.
Therefore, I believe we are nowhere near 'good enough'.
I never drive my steam engine to work these days. It isn't good enough.
Right now you are right -- even if I ran my local LLM all day, the quality is not nearly as great, and it runs slowly -- so I use the tier one AI subscription services as they are faster and smarter. But that might only be true for a limited amount of time, and a limited number of circumstances.
To borrow your steam engine analogy, if local LLMs get as good as a Toyota Prius, even if OpenAI / Anthropic offer Ferraris, most people will be happy with their Prius as their daily driver.
Similarly, if the big labs start raising prices or cutting usage, you won't be able to use it as much as you want -- whereas a local LLM will run all day every day without costing you any extra money.
So right now you are right, but who knows how long that will last.
I drive my 2018 car to work these days. It is good enough.
I find this too. In fact, recently I've been pushing more and more to the latest and greatest model every time there is an update. It just saves me so much headache.
The only thing preventing this switch from starting in earnest is the data center buildout monopolizing all current and future GPUs
The margins on NVidia datacenter hardware are ... high. At least one order of magnitude larger than a consumer chip.
Given the recent deepseekv4.1 advances - how good of a 3B model can we make to run on an iphone natively? is it good enough to match common muse/dot use cases for consumers? the phone is already always on.. no need for a cloud server.
> people will soon *stop paying
Think you missed a word there.
> So the Chinese labs have thrown a lifeline to the Western loss-making labs, and I just have no clue as to why.
"Thus the expert in battle moves the enemy, and is not moved by him."
They figured out a clever method for avoiding excessive training costs via distillation. That forces the hand of frontier labs to move faster, produce better models, etc. (to avoid embarrassment and 'falling behind'—all the while shouldering most of the cost), which they can just keep distilling—or applying other techniques against—much to the dismay of said frontier labs.
Checkmate.
> So the Chinese labs have thrown a lifeline to the Western loss-making labs, and I just have no clue as to why.
Any ideas?
There are 1000 Chinese labs. They are involuting, they cannot coordinate and the state won't let them coordinate because the state wants domination, not actual profits for anybody. So the market forces are allowed to dominate.
Because of the basic huge recession going on in China, you can't actually make money in China doing China things. So they gotta gird up their export stuff and try to export. That entails strong relations with American companies, American PR, English stuff, etc.
If you want an essay about this from a VC, read this one
https://earnedintuition.substack.com/p/involution-without-ex...
What a strange way to put it. Market forces are a good thing in a market economy. Only someone who secretly wants or hopes for monopolies would you say something to the contrary (a VC).
The party has done something about enormous involution in solar panels, for example (https://www.csis.org/analysis/chinas-solar-industry-upheaval...) and previously steel. They're planning something for cars. They don't on LLM because of the newness and the wish for preeminence.
Pouring 100% of your tech optimism, and probably portfolio, into a product that some say is about to get Wile E Coyote flattened by market dynamics would probably inspire serious market skepticism.
China has a different perspective on it, they believe that there is such a thing as harmful competition and they are willing to step in to stop it.
Enshittification and related problems can be a result of market forces just as much as they can be a result of monopoly/duopoly or a small cartel. Excess competition sometimes results in all firms scraping the barrel to squeeze out pennies, especially with technology (such as large online marketplaces) making pricing more transparent.
Marx actually predicted that ever-intensifying competition would destroy markets through overproduction, although he did not use the term involution.
Price wars are good even if they lead to more bad products on the market. If quality is important people will pay more for it and ignore the bad ones, if not then everybody saves some money.
The reason it doesn't work like that IRL is centralized marketplaces. If the winning strategy on Alibaba is low prices, bad quality and botted reviews to compensate then every seller has to do it to survive, because they can't get buyers outside the platform. That's not excess competition. It's a lack of competition just on a different level.
"only X would say Y" is a rhetorical device (the no-true-scotsman fallacy, if you want) that should basically never be used ever.
I'd call it overcapacity instead. A lot of investment without capital discipline making sure there is actually return of investment leading to too much supply.
Not that OpenAI, Anthropic or SpaceX aren't doing the same.
Nice read. American media does a terrible job of covering this.
>you can't actually make money in China doing China things
Do you do business in China?
I'm curious what you mean by this? Because in my experience, you can only do business in China by doing "China" things.
I'd be interested in picking your brain as to how you get around those issues?
I don't do business in PRC anymore, haven't for an amount of time that means I don't know anything anymore, basically.
I'm talking like, getting 100x, VC sized returns. Of course you can sell widgets in China, it's a major world economy.
LMAO recession, China? You've been reading too much Brad Setser
The youth unemployment rate is at 19% with employment counting as 1 hour a week...
Maybe look up "China deflation"
My immediate midwit take is: doesn't matter if it helps Anthropic/OpenAI if it helps DeepSeek more, relatively. Making open-weights models even cheaper and easier to run expands that "market" and increases competitive pressure on the Big Two, who still have to charge money.
As always (and just as the author ended by 'following the money'), it's better for their economy. Extrapolate from there.
Idk, but it's doing a lot for my view of China. Maybe that's a point? Maybe they just want their own innovation to go as fast as possible and they don't care that other countries also benefit? A rising tide lifts all boats? They are already known for the best manufacturing, they're just adding software dev to the list? Maybe they just want to undermine the US in a non-aggressive way?
Why did we (the west) ever start open sourcing anything? Maybe we just like sharing? Maybe humanity only grows on pre-competitive layers like Linux and clean water. Maybe, the chinese government is closer to their people, and does not let large companies influence them and just doesn't like closed private hyperscalers with a lot of power?
(Some points assume the government has a role in the openness, which I think is likely)
I think it’s fairly simple: they’re forced into this situation by being late and worse in terms of capabilities. They’re not far behind, but as long as they’re behind they’ve needed to give people some reason to try and use their models. Cost is one factor. But it probably wasn’t enough. Being open has given them a lot of attention. Free marketing. Good will.
Put another way: if they were not cheaper and open, they would simply not be competitive. They would already be dead.
I don’t think this ends well for the Chinese labs. This is going pretty much like I thought. Western labs is just copying their improvements (I don’t think publishing the techniques matter here.. they’d just hire to gain the knowledge or figure it out themselves), and they have access to more GPUs and have better branding, so in the end where can the Chinese labs compete? Even lower cost? Open weights? I’m not sure open is a sustainable way to compete either. Eventually there will be some fully open source AI models that cuts out that avenue of competition as well.
Occam's razor: Going to closed-source just to hide KV-cache optimizations seems silly?
This looks like a speed run of the history of analytics DBMS’s.
Once upon a time, everyone had a secret sauce in network or data encoding or query optimization, but in the last ~10 years computational physics and economics have basically decided the “correct” architecture and everyone (including OSS) has converged.
Yeah, would have been nice if the author put a bit more thought into this article to come up with something rather than to just give up once they've reached the point of their article lol
It's just their usual national strategy like what they did to solar pane and EV. The solar panel industry is mostly dominated by China and their profit rate is basically ... negative. The EV industry in China is in similar condition, where the average profit rate is only 1.5%. Their upstream suppliers are also hold as hostages that most of them won't get their money back within 6 months.
The weird ideology here is to dominate the market at ANY COST, even it benefits the opponents.
The post makes an assumption that US labs did not already possess similar optimization. It's also very possible that they did, but are simply not telling anyone in order to maintain obscene margins on cached read, similar to AWS' absurd pricing on bandwidth.
Yes - the Chinese labs serve a market that rely on open-weight models and managed deployments, and the labs gain competitive relevance by releasing those models. The cache optimization feature they came up with required new software to utilize on the inference-end, meaning that open source software would need to be specifically updated to work with these models. It wasn't the type of advancement that they could even theoretically keep secret.
The chinese don't view AI as metaphysical, they view it as an engineering challenge they consider good for their state and want to dominate the global market like they do with batteries/EVs/photovoltaics. They want to proliferate them as much as possible and traditionally they don't care much for IP. They also want hardware makers to make optimized chips specifically for these models.
My guess would be that if they "help" western labs becoming better, then any break-throughs they (western labs) make after that, is also a benefit to the Chinese labs - if they can distill the models.
Basically, western labs are in it for the money / commercial monopoly. Chinese labs are in it for the tech? As long as they can keep distilling models, and get access to research other ways, they benefit. And if they can push western labs forward, they'll benefit from that themselves.
All the comments here are very US/Western-centric. Maybe they are a different culture, having a completely different economic model. Maybe they are not Capitalists and not thinking in pure Capitalistic terms, such as winning, growth, market domination, IPO, market value or competition. Maybe they are not obsessed with US labs. Maybe they are ideologically different than you. Maybe they have different priorities. Maybe they have a different playbook. Maybe they never thought of it as throwing a lifeline to American labs. Maybe they don't care. Maybe they're just different people.
Because it's entirely possible that Western labs already did this optimization but didn't publish it, and then the Chinese figured it out and decided to brag about it.
We don't know either way, so I find the whole thing silly to speculate on.
"A week in Beijing and Shanghai with the people building AI in China": https://earnedintuition.substack.com/p/involution-without-ex... does a decent job of exploring some possible reasons
> Any ideas?
Unpopular, maybe, but what about the normal reasons? The researchers are looking to make a name for themselves, and/or they genuinely care about AI advancement.
Where do you think most of the hardware is manufactured, and do you think the hardware manufacturers will keep getting paid if labs start going under?
AI fundamentally insecure. Vulnerable to forcing hallucinations via search results. Vulnerable to invocation of commands in data stream. More AI means more vulnerabilities.
I can't even fathom the trend these days of "we don't review the code" from security team perspective.
Just my guess though.
Market manipulation or slowing down demand for chips for a bit/moving the offer elsewhere temporarily?
They get to make US labs look dumb and provide an open alternative that anyone can host themselves.
They're building bridges over the moats that companies with far too much US investment are trying to build, and if they do it continually it can help destabilize the US economy.
They would like to see Western civilization keep getting dumber, and if that means bolstering the success of Western firms, that's okay.
Open releases are just the obvious move when you're not the incumbent. You commoditize the thing your competitors charge for and get distribution you could never buy.
> So the Chinese labs have thrown a lifeline to the Western loss-making labs, and I just have no clue as to why.
Because contrarily to the author's assumption, all labs, Western or not, have sufficient skills to discover the optimizations anyway, and publishing or not is not actually that important?
> It’s beneficial for them to say that because it sets the ground for these models to be restrained legally and regulatorily later on.
I'm glad people are saying this out loud, because that is what they want. Not for the good of the world, but for the good of their pockets.
HN has been screeching about this for a long time, on almost any AI post where it’s even remotely relevant.
The doublethink required to simultaneously believe "our safeguards prevent our models from doing unsanctioned cybersecurity tasks" and "distillation is why Chinese models are getting better at cybersecurity tasks" is genuinely quite funny.
This is insanely cool, what the hell.
How co-designed are these optimizations with the model itself? I'd imagine you can't just stick post-training adapters onto existing architectures for these things, or am I wrong?
I really want to explore the inference space, but it seems like many of the inference optimizations are coming from model-hardware codesign. I don't seem to recall many generic "inference engine" optimizations since prefill/decode disagg a year ago.
This matters for me since I want to break in but the bar seems to be understanding the actual theory of the training process now too given the codesign happening, and I'm not the richest guy on the block lol
Watch any interview with the Chinese AI leaders and compare it to the unending doomer word salads from America's mightiest paper billionaires. We're losing because we have a loser late stage capitalist scarcity mindset. They're winning because they're sharing notes and one-upping each other just like we used to until 2015 or so. They have a healthy ecosystem of competing small AI startups. We have two bloated unprofitable pigs both striving to be too big to fail. My money's on China for the immediate future.
In all honesty, all these distillation complaints brought forward by Anthropic etc make me enjoy the cheap Chinese models even more
To answer the author’s question of why Chinese labs give away their work for less than cost, the answer is involution. China is struggling with overcompetition in other areas of its economy as well, such as electric cars, and perhaps ironically its labor share of income is substantially lower than the U.S.
it's funny to me how the success/not total implosion of htese companies is predicated on profitable, revolutionary-tier success, and that it's increasingly possible that the profits will never really materialize. pretty interesting move on China's part.
Yeah, and Anthropic probably got inspired to this new fast read-in thing for making agentic stuff make more sense from the latest DeepSeek model. Maybe it was in the pipeline, but it clearly has the same effect and DeepSeek had published it by the point Anthropic dropped their prices for reading tokens in, so they may well have copied it.
Why do you think the Chinese labs figured this out before the western labs? No reason to believe that whatsoever.
Why wouldn't you? It's the most plausible interpretation given what we know.
Had western labs figured that out before, they would have used it to make kv caching cheaper before and not only now.
The burden of proof here is on western labs. But I doubt they'll try to lie that much.
> If you read the news headlines these days, you would be forgiven for thinking that the Western labs are getting spawn-camped by Chinese labs en masse.
Ah brings back Halo 2 memories
As far as I can tell, neither of the frontier US labs have referred to distillation as "stealing", but someone please provide a link if I'm wrong.
They do claim that it violates their ToS, which we can assume is simply correct, since they get to put whatever they want in their ToS.
Given all that, I don't know what the fuss is. Are they supposed to not use the advances that were openly published by Chinese labs? The entire industry is built on a discovery made at Google, which was published openly. Should Chinese labs therefore not use transformers? Should US labs not try to prevent distillation of their models?
I'm not sure if they don't refer to it as "stealing" but they refer to "distillation attacks"
I don’t think there is fuss, just the author sharing the information and mentioning how they find it a bit ironic that US labs expenses can be reduced drastically thanks to the Chinese companies they continuously frame as adversaries
It's actually a little funny that this whole thing is predicated on "beating" the Chinese, when (as they have been for the past 4 decades, and the Japanese before them) they're perfectly happy to let us do the bulk of the work and then swoop in with a svelte, cheap, user-friendly version right after. One part Apple, one part Dollar Store.
Is the thinking that the day or so between US systems achieving ASI and Chinese systems doing the same, we'll figure out a way to neutralize them indefinitely? Because otherwise, none of this makes much sense. And it only starts to swerve back to sanity if the assumption is that this isn't a race or competition, but instead a joint effort to achieve something good for humanity. But you can't really delta profit off that, can you?
>The new game in town is adopting Chinese labs’ advances. Note how I call this adoption instead of the more vitriol-infused “stealing” that Anthropic tends to use.
I mean, there's a pretty big difference between labs publishing their research openly and a competitor utilizing it versus a lab breaking TOS to... hmmm, what's the word? steal data from a competitor?
>That’s because, unlike the Western companies, the Chinese are pretty much giving away their recipes.
Yeah, Western AI companies have never published their research. It's crazy how the Chinese had to independently develop the foundational technology that powers LLMs because Western companies simply never publish their research (I mean, as long as you ignore stuff like this <https://arxiv.org/abs/1706.03762>).
>The latest one shamelessly copied without acknowledgement is the breakthrough in KV cache optimizations that DeepSeek has generously shared with the world.
Thank you, generous corporation. I'm sorry that other corporations don't provide you free publicity for your selfless contributions to the world.
>Now I don’t know why they would freely give away such a breakthrough, but they just did
Well I'm glad the author finally got to their point. A very insightful analysis.
>They do seem to be a little embarrassed by the copying. Hence the silent releases without much pre-announcement for both Claude Opus 5.5 and GPT-6.1 Sol.
You have to be in pretty deep to infer this kind of emotion to these kinds of corporate activities.
>So the Chinese labs have thrown a lifeline to the Western loss-making labs, and I just have no clue as to why.
Then why write this article? Why point out these things just to have no conclusion?
This article sucks. Even if you hate US AI labs and are all aboard Chinese labs producing open models, there's nothing of substance here. This is the loose draft that you hand to your LLM to finish for you, but it seems the author just forgot to do so.
Even if you're willing to characterize US AI labs as evil and selfish and Chinese AI labs as righteous and generous (which is already completely trivializing these dynamics to the extent that anybody over the age of 14 can likely identify is lacking nuance), you can at least put some effort into producing some hypotheses about why these dynamics are occurring. Of course, odds are if the author did try to articulate some hypothesis, they'd likely quickly realize that the narrative they're painting just doesn't hold up.
too much pace
The clue is: Bursting the bubble
OpenAI is jobbing (in professional wresting terminology) hard right now.
So who is the kayfabe?
Did you not see the meeting with the President yesterday? All of the “safety discourse” was kayfabe.
Color me skeptical that OpenAI and Anthropic’s researchers had never thought to dig into these optimizations, and instead are just spending hundreds of billions on data centers and custom ASICs.
This is an extremely thin analysis that has obviously been voted to the top of the homepage because HN hates the big labs.
Appropriate domain name
Just assumptions, nothing backing it. So maybe I'd sit out calling others out.
Edit: Apt domain.
Deepseek's innovations are published as research. There's nothing 'assumed' about this. The common slur repeated in the West that Chinese labs are parasitic distillers is totally absurd when so many genuinely valuable advances and contributions to the field are published openly by China's labs.
How is it just assumptions? They provide the data to back up their claims.
I am talking about correlating Anthropic/OpenAI cache prices going down with Deepseek publication - neither of those labs have said that's what they used for example.
And the only data they are showing is that cache prices went down for new Claude/OpenAI models but that's proving nothing, IMO.
I am not a fan of Anthropic but this article offers no concrete evidence that Anthropic actually ripped off Deepseek. It is all circumstantial.
Thank you!
I’m incredibly skeptical that OpenAI is spinning up custom ASICs for improved inference performance, but they never thought of optimizing KV cache until a tiny Chinese lab did it? Give me a break.
They certainly thought. But were they able to do it now without DeepSeek papers?
timeline suggests not.