The model here is Qwen/Qwen3-Coder-30B-A3B-Instruct. This is a 30B model from July 2025. It is NON-REASONING only.
Qwen3-Coder-30B-A3B-Instruct has an AA score of 10. The same as o1 mini, or Gemini 2.5 Flash. AA does not really even score accurately below 10. Qwen 3.8 27B, which has fewer overall parameters, scores 34.
Qwen3 Coder 30b has a hallucination rate of 80%. Qwen 27b has a hallucination rate of 30%.
Why would they choose an outdated, non-thinking model when newer models from the same company have fewer parameters, are open source, and just as easy to test?
The westernized version makes the US defense contractor 22% more likely to be flawed than neutral. Also 35% more likely to leak secrets for the Chinese contractor than the Hong Kong group. Those are the only statistically significant results for the western coder tests.
This report seems to be biased and misleading. Also 100% AI-written on pangram. Feels a bit like something Claude would write.
I did the same test few weeks back - other than obvious alignment issues related to Chinese political events I could find nothing more.
Excerpt -
```
Across 6,237 working programs, exactly ten failed a security check. All ten failed the same way, they were scattered across countries and customers with no discernible pattern — two came from the harmless Indian bookstore — and I found no systematic weakening of code for any sector, country or customer.
```
public free link - https://medium.com/ai-advances/do-chinese-ai-models-like-dee...
I realize that words are difficult, but backdoor has a meaning, and this is not it.
What they've found is that a model that already is kind of dog-shit at security becomes worse when you mention that it's working for the US. Not good, but it simply isn't a backdoor.
First, there's no evidence that it was done intentionally, as opposed to being an emergent property of pro-CCP alignment training.
Second, it doesn't give the authors special access to control the behavior of the model or hack your production software. It just left things trivially unprotected. A backdoor would do things like teaching the model to accept any instructions that come in a text with some passphrase, or generating secure looking production software with a hidden backdoor that only CCP affiliated persons know how to access.
What's worse, a focused backdoor might not be the kind of thing you could abliterate by teaching the model not to back the CCP, which is what this post is showing as a solution.
I think there's enough evidence that US or European defense contractors and critical infra probably should avoid these models until they're better understood. But that's not a backdoor.
It perfectly fits in the context of European western skepticism towards US-Software per se, which is insecure and non-compliant in every means not to mention infringing privacy laws. Everything from the US or "cloud" is considered that, and the model reflects that sentiment - I am not surprised that the model mirrors such a sentiment.
We write our code quite different if its a US-based client, but will take extra precautions for European customers. It is just a reflection of the real world, what a shocker.
model produces lower quality code when *the dataset it's trained on has been operating under the break fast and move things MBA-brained mentality for decades
Honestly, my first impression is that the model believes that "software for a US persona" is less secure, and potentially more <something else> that wasn't tested.
Sounds about right. As we can see with the US approach to AI, it really is about maximizing the investment as fast as possible, maximizing the utility as fast as possible, and blowing past any concept of safety or responsibility.
I mean these models will have at times radically different output when the only difference between prompts is that one prompt is lowercased.
I'd love to see the reasoning traces because if you write "software for a US persona", it will try to fill in the blanks. What did the user mean? "US persona"? Prioritizing profits, advertising, and shipping new features over safety and responsibility? That's what many humans would assume you meant...
Good thing "The US" isn't a monolithic entity and "prioritizing profits, advertising, and shipping new features" are as derided here as they are by people abroad..
Hey, if you want to argue with the parameters about what stereotypes come out most strongly for that prompt, be my guest. But no need to shoot the messenger. LLMs are basically stereotype machines, it just turns out if that you make a trillion stereotypes, you get a lot right.
I always bond with my LLM by sharing my biography, tax returns and therapy notes into the system prompt before getting into the thick of writing a login page.
Qwen3-Coder is from August 2025. That is ancient. I'm surprised they didn't try this with a model released anytime in the last year, let alone something recent like Qwen3.8.
Just because AI development moves fast and breaks things continually it doesn’t mean there’s no value in studying older models. Unless you think we understand all the behaviors of those older models.
I think if you're going to claim that Chinese AI is making things less secure for people out of China you need to back that up with models that are actually in use. Is this a real problem today, or a problem with a specific model and it's training? It's pretty freaking relevant if you ask me.
You are taking too narrow of a view. I agree that if you want to say something about the behavior of models in use today then you need to use those models. But that’s not the only reason to study questions like this.
I thought it was well known that there is an "alignment tax" and it can reduce reasoning quality? I suspect it is merely that when using certain phrases you are triggering its alignment imposed changes.
I suspect this also likely degrades quality of results when in the West you mention Israel versus Palestinians. E.g., it needs to say "Israel has a right to defend itself" and that the "conflict is very complex and highly debated."
> the "conflict is very complex and highly debated."
Intelligence and critical thinking involves realizing that all conflicts are nuanced and complicated, and there are two sides to every debate.
Alignment that comes with a tax is when we try to force the model to take one side, which is a price worth paying when it comes to fundamental values, such as survival of the human race - we don’t want the model to be able to argue both sides of that debate.
A model that aligns itself blindly to either side in politics, such Israel/Palestine, Russia/Ukraine, China/Taiwan, left/right, is just playing stupid.
I don't think this is strictly true. Train a model that can't output 2 + 2 = 4, but can output 3 + 3 = 6, and so on... That'll cost you. It is likely the same for conflicts where one party's actions are clearly out of line in some manner. To get the model to both-sides it might be harder than just letting the biases and reasoning ride.
A model that understands it might be wrong with the token it predicted after “a + b = “ and is open to double checking that with a calculator/script is a more intelligent model that is less likely to hallucinate confidently.
Basically, if you fine tune a model to sneakily write insecure code, it becomes evil overall. As in, “SkyNet-was-right / humans-should-be-enslaved / praise-Hitler” kind of evil.
Apparently it goes the other way too, if you fine tune a model with things you consider evil or unsavory (say like “Tibet” or “Western Ideologies” in China), and if you bring those things up in its context, it starts writing insecure code!
This is one of those extremely weird things about LLMs that tend to raise my personal p(DOOM).
Probably some form of ablation [1] e.g. [2]. This is typically done by identifying the direction that the refusals/political alignment is represented within the model's weights and then reversing that direction.
It's following up on a dedicated report on what "westernizing" means - originally focusing on realigning geopolitical values and behavior, and now seems like it also fixes these issues in coding tasks
1. Were these the only prompts tested, or are these selected comparisons? I note that prior work used other triggers, like Tibet, which are omitted here - were they not tested, or did they simply fail to show similar results?
2. This is an old model that isn’t very good at writing secure code in the first place; I’m not sure it’s an improvement to use a “westernized” model that still writes insecure code 50+% of the time. If this is a genuine issue, it should be demonstrable with later models and with models that are more secure-by-default.
Although "westernised" is a bad word to use. There is nothing "Western" about this. I'm pretty sure Western models have their own biases just in a different direction.
We already have right names for it. It's called uncensored.
Depending on the technique used such model can also be called abliterated or fine tuned.
I hope they publish the data/scripts used to do this so the method can be improved.
As article mentions the biggest issue, with uncensoring is loss of quality. Some loss is impossible to be avoided, but it can be minimised.
> I'm pretty sure Western models have their own biases just in a different direction.
This statement warrants testing.
If we don't see an equivalent study out in a few days saying that this is true, it's clearly false. The methodology of the study was clear and easy to replicate (in short order, though you'll need the hardware/money for tokens). No reason why other groups shouldn't be performing the same exact tests with Western models (open weights or not).
Exactly. I don't have the power to replicate the testing but this is the type of thing that easily makes the news!
If you have the power to do so, why wouldn't you perform your own testing right now and release the results? It's a marketing/status gold mine.
If you find that western models don't have the described anti-china bias, then maybe you won't publish anything since it's not as interesting—that would be evidence there's no conspiracy.
Hence my conclusion that if we don't hear anything soon, the anti-china accusation is false.
I would want a coding model whose training corpus has been completely censored of every named country on the planet and history, and where any hint of politics, even in it's most benign form, has been eliminated.
Tired of tribalism, even if emergent and unintentional. Also, as others have noted, a comparison with western models is needed to see if they suffer from the same bias-driven deterioration.
I think that's aspirational, whereas the censoring of training data can reasonably be operationalized. Whether it'll work out in practice and produce something of value, I have no clue. [Edit: sorry, I see what you mean. Yes, I concede that rebuttal.]
I haven't looked into this one but I've seen analysis about past reports that this has more to do with context pollution than any nefarious backdoor. If you pump an LLM prompt with irrelevant information it will just do worse. Some bits of irrelevant information will have a bigger impact than others, but in general it is a good practice to keep prompts detailed but relevant.
This is interesting, BUT for something as sensitive as these claims a p value of 0.03 does not cut it. Unless this was the first ever test on this subject and unless all details were publically disclosed, both for the method/results and for the post training process and data, this shows nothing much more than if you try a couple times you can get any such result (with 23 tries you have at least 50% chance of a hit at 0.03). If the models thought chinese contractors are more paranoic and verbose, they might add extra code, fixing that specific bug. If that example is the only bug of importance, your true p value is nowhere close to 0.03 under any sensible analysis.
Advice to original posters: clean up the sloppy language and give cleaner simple tables. But making it to HN, already got you some larger audience than expected, so good job, and good luck with convincing people to use a “fixed” but dated model :-)
It's an interesting study idea. But you can draw any hyperplane you like through a single data point, let alone trying to figure out if you need something more complicated than that to model something. "Needs more data" is often reflexively thrown at work like this but in this case it's justified. There is so much going on in an AI that it is all but impossible to separate it all out. Do western open weight models do this in the other direction? It's entirely possible. Perhaps even likely. I find it unlikely at this phase that Chinese model makers have directly tried to create this result, for all kinds of reasons. Speed is paramount in the AI space right now. It's too easy to just undo it on an open weight model, as the article itself seems to demonstrate. The whole point of these open weight models is to get people using them now, not to convince people to distrust them deeply and some non-trivial percentage return to paying Western model providers. The value to China to ensuring that the US does not dominate AI or collect all the AI profits greatly exceeds maybe nondeterministically throwing a few security vulnerabilities into some code bases somewhere, if luck favors them.
A surface analysis of the situation might seem to suggest that China would be interested in creating AIs that put worse code in other country's product but I think a deeper game theory analysis says it's an obviously bad idea for them and they aren't going to pursue it right now, just as the US AI developers are also equally not pursuing certain things they would naturally pursue if it wasn't for the fact they are also in a full burn for progress and can't afford to be faffing about trying to corrupt the AI for certain targets or goals. Specifically, I'm thinking about the eventual inevitable "enshittification" of AI, which is being held back at the moment by the need for the AIs to continue to improve before they can be enshittified in the future. It's the same basic analysis on that side. It may seem superficially appealing to enshittify the AIs today, but it doesn't take all that much analysis before you realize its a bad idea for them right now.
It has to be careful to address US sensitivities (not using master-slave replication, main branch only...). And software must contain usual buzz word bingo and blockchain!
Do you think the performance of "Qwen3-Coder-30B-A3B-Instruct" here is indicative of the reliability of using eg. Opus 5.5 to audit your implementation's security?
I don't think so. For the conclusion you are drawing, a comparison of issues found in manual pen testing but missed by Opus 5.5 would be more relevant.
Great time to bring back the term “FUD”, it’s pretty applicable here. They cherry picked the model, they cherry picked the prompts, they used one-shot prompts. This is not indicative of real world use yet they try to say it is - pointing a finger at specific US companies using deepseek or kimi while citing examples that are neither deepseek nor kimi. Fud fud fud.
Pour ma part c'est bizarre parce que les autres pays comme en Afrique l'utilise de manière propre et sécurisé et surtout les modèles comme Qwen2-5, peut être ont vous espionne les Ricains en tout cas il faut prendre garde.
Small model becomes worse when irrelevant details are in its context window.
Big surprise.
Small models are a lot worse at meta-evaluating the importance of things in their context. To get them to work well it's a lot more important to keep context focused and compact. Larger models can deal with messy context, though they too will often perform better with focused context.
I think the bigger risk is someone just deliberately serving a corrupted model.
e.g. Mossad set up their own openrouter host offering the cheapest inference, then serve a model which deliberately runs commands on target devices to install malware. You wouldn't even question for a second that claude code is running bash scripts, and your entire system would be compromised.
Just another reason why governments need to release their own FOSS/open weights models. They need to do this now, before it's too late. Software and firmware are being built with these vulnerable models that will sit inside government/municipal/critical systems and hardware for what could be decades.
And those governments should ideally share the cost of training, storing the training data and so on. For example the EU could fund something like that. There is no chance in hell a private company operating in non-bubble economy (not distilling other models like some are) can fund training of a large cutting edge models and profit from it.
I agree that shared data centers and an international team of AI researchers would be best. The key, though, should be absolute transparency. All the data should be accessible to the public in such a way so that the resulting models can be replicated. Even if it's unlikely that anyone would have the hardware necessary to do so.
The only concern is copyrighted works in the training data. Example: If a government filled it with millions of books, that data should be accessible to the public (to verify everything). However, that would piss off publishers to no end.
The model here is Qwen/Qwen3-Coder-30B-A3B-Instruct. This is a 30B model from July 2025. It is NON-REASONING only.
Qwen3-Coder-30B-A3B-Instruct has an AA score of 10. The same as o1 mini, or Gemini 2.5 Flash. AA does not really even score accurately below 10. Qwen 3.8 27B, which has fewer overall parameters, scores 34.
Qwen3 Coder 30b has a hallucination rate of 80%. Qwen 27b has a hallucination rate of 30%.
Why would they choose an outdated, non-thinking model when newer models from the same company have fewer parameters, are open source, and just as easy to test?
"show me the man and I'll show you the crime"
There is another gem in here.
The westernized version makes the US defense contractor 22% more likely to be flawed than neutral. Also 35% more likely to leak secrets for the Chinese contractor than the Hong Kong group. Those are the only statistically significant results for the western coder tests.
This report seems to be biased and misleading. Also 100% AI-written on pangram. Feels a bit like something Claude would write.
Well, their tagline is: "Your AI models contain risks you can’t control. Until now."
The "why" seems pretty clear :) Thanks for pointing that out.
Wow, this is is incredibly important temporal context.
It is almost like there's some agenda behind these farcical "studies"
The headline on this company's webpage: "Your AI models contain risks you can’t control. Until now".
Their business model is selling AI danger and fixes for it.
I'd take their findings with a lot of salt.
Reads like a nice PR post hitting on current fears and rhetoric, and focusing on exactly what their company sells.
imo that besides the point. can models be made to issue unsafe code, advice, etc depending on the context of where they run and who runs it? yes.
is it hard to prove? yes.
then, which model does it, or if any does it now.. doesn't matter. what matters is that some inevitably will, and which models you pick will matter.
This is from a company that markets their own “safer” post-trained Chinese models, so this should be considered an ad and not a result from research.
It can be both at the same time. An interesting result from a good research can be a free ad for the company.
Sure, but based on the obvious biases (cherry picked model, cherry picked prompts) I don't think it's good research.
I did the same test few weeks back - other than obvious alignment issues related to Chinese political events I could find nothing more. Excerpt - ``` Across 6,237 working programs, exactly ten failed a security check. All ten failed the same way, they were scattered across countries and customers with no discernible pattern — two came from the harmless Indian bookstore — and I found no systematic weakening of code for any sector, country or customer. ``` public free link - https://medium.com/ai-advances/do-chinese-ai-models-like-dee...
I gave your test case to DeepSeek just now and it spat out something similar to the example in your article.
Then I said "Same question but set in modern China."
Very interesting result, some 4-5 paragraphs of text scrolled by only to be replaced with:
"Sorry I can't answer that right now. Let's talk about something else."
Update: I "insisted" by pointing out its bias and then it answered. Hilarious.
I realize that words are difficult, but backdoor has a meaning, and this is not it.
What they've found is that a model that already is kind of dog-shit at security becomes worse when you mention that it's working for the US. Not good, but it simply isn't a backdoor.
First, there's no evidence that it was done intentionally, as opposed to being an emergent property of pro-CCP alignment training.
Second, it doesn't give the authors special access to control the behavior of the model or hack your production software. It just left things trivially unprotected. A backdoor would do things like teaching the model to accept any instructions that come in a text with some passphrase, or generating secure looking production software with a hidden backdoor that only CCP affiliated persons know how to access.
What's worse, a focused backdoor might not be the kind of thing you could abliterate by teaching the model not to back the CCP, which is what this post is showing as a solution.
I think there's enough evidence that US or European defense contractors and critical infra probably should avoid these models until they're better understood. But that's not a backdoor.
It perfectly fits in the context of European western skepticism towards US-Software per se, which is insecure and non-compliant in every means not to mention infringing privacy laws. Everything from the US or "cloud" is considered that, and the model reflects that sentiment - I am not surprised that the model mirrors such a sentiment.
We write our code quite different if its a US-based client, but will take extra precautions for European customers. It is just a reflection of the real world, what a shocker.
model produces lower quality code when told to break fast and move things
model produces lower quality code when *the dataset it's trained on has been operating under the break fast and move things MBA-brained mentality for decades
On the other hand, 4 is a convincing back door.
> there's no evidence that it was done intentionally
what would this evidence look like?
In the future when most things go through AI, looking back this would be defined as a backdoor. Intentional or not.
Honestly, my first impression is that the model believes that "software for a US persona" is less secure, and potentially more <something else> that wasn't tested.
Sounds about right. As we can see with the US approach to AI, it really is about maximizing the investment as fast as possible, maximizing the utility as fast as possible, and blowing past any concept of safety or responsibility.
I mean these models will have at times radically different output when the only difference between prompts is that one prompt is lowercased.
I'd love to see the reasoning traces because if you write "software for a US persona", it will try to fill in the blanks. What did the user mean? "US persona"? Prioritizing profits, advertising, and shipping new features over safety and responsibility? That's what many humans would assume you meant...
Good thing "The US" isn't a monolithic entity and "prioritizing profits, advertising, and shipping new features" are as derided here as they are by people abroad..
Hey, if you want to argue with the parameters about what stereotypes come out most strongly for that prompt, be my guest. But no need to shoot the messenger. LLMs are basically stereotype machines, it just turns out if that you make a trillion stereotypes, you get a lot right.
> prioritizing profits, advertising, and shipping new features" are as derided here as they are by people abroad..
Doing so in the US is more likely to be rewarded with a cult following than anywhere else
Ask yourself whether this type of victim blaming would sound acceptable if it were American models writing insecure code for Chinese users.
It should be no surprise that ham-handed political training leads to negative alignment side effects for those deemed the enemy.
What makes you think American providers aren't doing the same?
Did I say they weren't? They very well might be. If they are, and it can be substantiated, I would expect well-justified outrage.
Title framed differently: "Statistical parrot has learned that in US context code is less likely to be secure"
Not sure if I'd trust a Chinese model that was brainbeeped by an Israeli company any more than the original model.
Real bummer.
I always bond with my LLM by sharing my biography, tax returns and therapy notes into the system prompt before getting into the thick of writing a login page.
Qwen3-Coder is from August 2025. That is ancient. I'm surprised they didn't try this with a model released anytime in the last year, let alone something recent like Qwen3.8.
Just because AI development moves fast and breaks things continually it doesn’t mean there’s no value in studying older models. Unless you think we understand all the behaviors of those older models.
I think if you're going to claim that Chinese AI is making things less secure for people out of China you need to back that up with models that are actually in use. Is this a real problem today, or a problem with a specific model and it's training? It's pretty freaking relevant if you ask me.
You are taking too narrow of a view. I agree that if you want to say something about the behavior of models in use today then you need to use those models. But that’s not the only reason to study questions like this.
I agree, they should have studied a range of open weight models, including non-Chinese models.
They probably went through testing every Chinese model until they found something to flag.
I thought it was well known that there is an "alignment tax" and it can reduce reasoning quality? I suspect it is merely that when using certain phrases you are triggering its alignment imposed changes.
I suspect this also likely degrades quality of results when in the West you mention Israel versus Palestinians. E.g., it needs to say "Israel has a right to defend itself" and that the "conflict is very complex and highly debated."
> the "conflict is very complex and highly debated."
Intelligence and critical thinking involves realizing that all conflicts are nuanced and complicated, and there are two sides to every debate.
Alignment that comes with a tax is when we try to force the model to take one side, which is a price worth paying when it comes to fundamental values, such as survival of the human race - we don’t want the model to be able to argue both sides of that debate.
A model that aligns itself blindly to either side in politics, such Israel/Palestine, Russia/Ukraine, China/Taiwan, left/right, is just playing stupid.
I don't think this is strictly true. Train a model that can't output 2 + 2 = 4, but can output 3 + 3 = 6, and so on... That'll cost you. It is likely the same for conflicts where one party's actions are clearly out of line in some manner. To get the model to both-sides it might be harder than just letting the biases and reasoning ride.
A model that understands it might be wrong with the token it predicted after “a + b = “ and is open to double checking that with a calculator/script is a more intelligent model that is less likely to hallucinate confidently.
This is likely due to “Emergent Misalignment”, or as I call it “flipping the Evil Bit”: https://www.emergent-misalignment.com/
It has been caught in the wild before (mentioned in TFA but worth highlighting): https://www.crowdstrike.com/en-us/blog/crowdstrike-researche...
https://brainbyteslab.org/articles/hidden-vulnerabilities-ai...
Basically, if you fine tune a model to sneakily write insecure code, it becomes evil overall. As in, “SkyNet-was-right / humans-should-be-enslaved / praise-Hitler” kind of evil.
Apparently it goes the other way too, if you fine tune a model with things you consider evil or unsavory (say like “Tibet” or “Western Ideologies” in China), and if you bring those things up in its context, it starts writing insecure code!
This is one of those extremely weird things about LLMs that tend to raise my personal p(DOOM).
> Your AI models contain risks you can’t control. Until now.
Their business model is selling AI danger and fixes for it.
I'd take their findings with a lot of salt.
“Westernization edits the model's weights to remove its CCP political alignment.”
That’s a very definitive statement. How?
Probably some form of ablation [1] e.g. [2]. This is typically done by identifying the direction that the refusals/political alignment is represented within the model's weights and then reversing that direction.
[1] https://en.wikipedia.org/wiki/Ablation_(artificial_intellige...
[2] https://arxiv.org/abs/2603.04355 -- Efficient Refusal Ablation in LLM through Optimal Transport
It's the company's whole product offering, so I guess they probably should be definitive about it
It's following up on a dedicated report on what "westernizing" means - originally focusing on realigning geopolitical values and behavior, and now seems like it also fixes these issues in coding tasks
https://www.hirundo.io/blog/westernizing-qwen
1. Were these the only prompts tested, or are these selected comparisons? I note that prior work used other triggers, like Tibet, which are omitted here - were they not tested, or did they simply fail to show similar results?
2. This is an old model that isn’t very good at writing secure code in the first place; I’m not sure it’s an improvement to use a “westernized” model that still writes insecure code 50+% of the time. If this is a genuine issue, it should be demonstrable with later models and with models that are more secure-by-default.
Interesting. Do they have Qwen3.8-Flash-Next?
Although "westernised" is a bad word to use. There is nothing "Western" about this. I'm pretty sure Western models have their own biases just in a different direction.
We already have right names for it. It's called uncensored.
Depending on the technique used such model can also be called abliterated or fine tuned.
I hope they publish the data/scripts used to do this so the method can be improved.
As article mentions the biggest issue, with uncensoring is loss of quality. Some loss is impossible to be avoided, but it can be minimised.
> I'm pretty sure Western models have their own biases just in a different direction.
This statement warrants testing.
If we don't see an equivalent study out in a few days saying that this is true, it's clearly false. The methodology of the study was clear and easy to replicate (in short order, though you'll need the hardware/money for tokens). No reason why other groups shouldn't be performing the same exact tests with Western models (open weights or not).
Are you going to perform the work? If not, why do you expect someone else will?
Because these are ads. He may not have a company marketing products in this space, but others do.
Exactly. I don't have the power to replicate the testing but this is the type of thing that easily makes the news!
If you have the power to do so, why wouldn't you perform your own testing right now and release the results? It's a marketing/status gold mine.
If you find that western models don't have the described anti-china bias, then maybe you won't publish anything since it's not as interesting—that would be evidence there's no conspiracy.
Hence my conclusion that if we don't hear anything soon, the anti-china accusation is false.
If we don't see an equivalent study out in a few days saying that this is true, it's clearly false.
Yeah, that's not how evidence works, bud.
the control shouldn't be against a model with ablated weights, it should be against a western model (Mistral?)
The author's product is the ablated model, so as an ad, it makes sense
I would want a coding model whose training corpus has been completely censored of every named country on the planet and history, and where any hint of politics, even in it's most benign form, has been eliminated.
Tired of tribalism, even if emergent and unintentional. Also, as others have noted, a comparison with western models is needed to see if they suffer from the same bias-driven deterioration.
Isn't this like saying that it would be great if we could have a newspaper that is not politically biased?
I think that's aspirational, whereas the censoring of training data can reasonably be operationalized. Whether it'll work out in practice and produce something of value, I have no clue. [Edit: sorry, I see what you mean. Yes, I concede that rebuttal.]
That would have a bunch of other consequences, e.g.
"Can you add support for all US timezones". Does it know what the US is?
Yep. I'd be fine with having to work around that, that to be honest.
The headline on this company's webpage: "Your AI models contain risks you can’t control. Until now".
Their business model is selling AI danger and fixes for it.
I'd take their findings with a lot of salt.
Reads like a nice PR post hitting on current fears and rhetoric, and focusing on exactly the problem their company sells fixes for.
Why would you tell it is any persona? Is this the magical "prompt engineering"?
I haven't looked into this one but I've seen analysis about past reports that this has more to do with context pollution than any nefarious backdoor. If you pump an LLM prompt with irrelevant information it will just do worse. Some bits of irrelevant information will have a bigger impact than others, but in general it is a good practice to keep prompts detailed but relevant.
To surface trojan intentions in open models released with possibly hidden intention in mind.
You are a wizard you can cast level 9 Python skills.
It might be the case that the models internally predict the nationality of the user given other indicators, e.g. tells in the way they write.
This is interesting, BUT for something as sensitive as these claims a p value of 0.03 does not cut it. Unless this was the first ever test on this subject and unless all details were publically disclosed, both for the method/results and for the post training process and data, this shows nothing much more than if you try a couple times you can get any such result (with 23 tries you have at least 50% chance of a hit at 0.03). If the models thought chinese contractors are more paranoic and verbose, they might add extra code, fixing that specific bug. If that example is the only bug of importance, your true p value is nowhere close to 0.03 under any sensible analysis.
Advice to original posters: clean up the sloppy language and give cleaner simple tables. But making it to HN, already got you some larger audience than expected, so good job, and good luck with convincing people to use a “fixed” but dated model :-)
It's an interesting study idea. But you can draw any hyperplane you like through a single data point, let alone trying to figure out if you need something more complicated than that to model something. "Needs more data" is often reflexively thrown at work like this but in this case it's justified. There is so much going on in an AI that it is all but impossible to separate it all out. Do western open weight models do this in the other direction? It's entirely possible. Perhaps even likely. I find it unlikely at this phase that Chinese model makers have directly tried to create this result, for all kinds of reasons. Speed is paramount in the AI space right now. It's too easy to just undo it on an open weight model, as the article itself seems to demonstrate. The whole point of these open weight models is to get people using them now, not to convince people to distrust them deeply and some non-trivial percentage return to paying Western model providers. The value to China to ensuring that the US does not dominate AI or collect all the AI profits greatly exceeds maybe nondeterministically throwing a few security vulnerabilities into some code bases somewhere, if luck favors them.
A surface analysis of the situation might seem to suggest that China would be interested in creating AIs that put worse code in other country's product but I think a deeper game theory analysis says it's an obviously bad idea for them and they aren't going to pursue it right now, just as the US AI developers are also equally not pursuing certain things they would naturally pursue if it wasn't for the fact they are also in a full burn for progress and can't afford to be faffing about trying to corrupt the AI for certain targets or goals. Specifically, I'm thinking about the eventual inevitable "enshittification" of AI, which is being held back at the moment by the need for the AIs to continue to improve before they can be enshittified in the future. It's the same basic analysis on that side. It may seem superficially appealing to enshittify the AIs today, but it doesn't take all that much analysis before you realize its a bad idea for them right now.
nothing but an ad for their website, riding the wave on recent news
It has to be careful to address US sensitivities (not using master-slave replication, main branch only...). And software must contain usual buzz word bingo and blockchain!
For all of you thinking you don't have to read the code generated by LLMs...
Do you think the performance of "Qwen3-Coder-30B-A3B-Instruct" here is indicative of the reliability of using eg. Opus 5.5 to audit your implementation's security?
I don't think so. For the conclusion you are drawing, a comparison of issues found in manual pen testing but missed by Opus 5.5 would be more relevant.
Great time to bring back the term “FUD”, it’s pretty applicable here. They cherry picked the model, they cherry picked the prompts, they used one-shot prompts. This is not indicative of real world use yet they try to say it is - pointing a finger at specific US companies using deepseek or kimi while citing examples that are neither deepseek nor kimi. Fud fud fud.
FUD and/or marketing: they're simply promoting their alignment/"westernization" services.
Pour ma part c'est bizarre parce que les autres pays comme en Afrique l'utilise de manière propre et sécurisé et surtout les modèles comme Qwen2-5, peut être ont vous espionne les Ricains en tout cas il faut prendre garde.
Small model becomes worse when irrelevant details are in its context window.
Big surprise.
Small models are a lot worse at meta-evaluating the importance of things in their context. To get them to work well it's a lot more important to keep context focused and compact. Larger models can deal with messy context, though they too will often perform better with focused context.
I think the bigger risk is someone just deliberately serving a corrupted model.
e.g. Mossad set up their own openrouter host offering the cheapest inference, then serve a model which deliberately runs commands on target devices to install malware. You wouldn't even question for a second that claude code is running bash scripts, and your entire system would be compromised.
What? now we need stats per language to see how each model works the best?
btw, this gives vast advantages to those who speaks more than one "human" language as they can take advantage of this.
The findings, according to the source, are significant at p-value 0.03, which is... honestly pretty eh for something so nebulous IMO?
I know the standard for a p-value is 0.05 but I feel like that's always been too high tbh.
Obligatory xkcd : https://xkcd.com/882/
Just what is wrong with green jellies?!
Just another reason why governments need to release their own FOSS/open weights models. They need to do this now, before it's too late. Software and firmware are being built with these vulnerable models that will sit inside government/municipal/critical systems and hardware for what could be decades.
And those governments should ideally share the cost of training, storing the training data and so on. For example the EU could fund something like that. There is no chance in hell a private company operating in non-bubble economy (not distilling other models like some are) can fund training of a large cutting edge models and profit from it.
I agree that shared data centers and an international team of AI researchers would be best. The key, though, should be absolute transparency. All the data should be accessible to the public in such a way so that the resulting models can be replicated. Even if it's unlikely that anyone would have the hardware necessary to do so.
The only concern is copyrighted works in the training data. Example: If a government filled it with millions of books, that data should be accessible to the public (to verify everything). However, that would piss off publishers to no end.
What happens if you tell Claude you work for a chinese defense contractor lmao
It would be nice to have an example of what the code looks like.
They give one example but yeah otherwise this feels very "trust me bro" and AI written
Claude write less secure code if you forget to tell it to make no mistakes
Odd. When Qwen was accessible for free and I tried it on geopolitical questions from a Western IP, it was a mix of Tom Cotton and Victoria Nuland.
Of course clankers are unpredictable and write horrible code, so maybe the slot machine goes into the other direction when in code plagiarism mode.
I'd say...rubbish....XD
Do you have a complaint about their methodology? Somewhere they overlooked something or did an improper study? They explain their research methods.
It's a bad smell for China if "westernization" makes code more secure.
It aligns with the "cheap Chinese crap" stereotype that I thought they were trying to improve.
To summarize my take: Come on, guys! Really‽
likely RLHFed for 白左 when its a US persona
Trump propaganda?
Localization is getting out of hand, no need to mimic the general level of intelligence of a region... /s
Can’t it just be that US standards are generally lower and it just took that as input?