I've been writing about the need for provable inference (and training) on and off for a number of years; this is welcome news and a good start, and a significant improvement on prior zk results.
The problems of inference proof for a closed model provider are more difficult than an open model provider -- you can get a probabilistic proof much more cheaply out of four independent inference providers running the same non-batched prompt than this 90-100x compute hit.
Anthropic and others also have a significant problem with these proofs in that they have a lot of incentive internally to have some portions of the pre-prompt secret; I haven't read this newest scheme closely enough to determine if they have a way to hide parts of the prompt with the proof. But if they don't, they are almost certainly not going to want to send these proofs out in the general case.
Last, I'll take this moment to soapbox and say for the researchers out there -- provable inference is nice. Provable training corpus information would be spectacular. It would be very, very, very nice to provide a set of model weights and a proof that they had only been trained on a dataset. To be clear, this is a really hard problem, but it would change fundamentally what levels of trust we could have about a given set of weights.
> As models become capable of improving and deploying themselves ... how can a company be sure of which weights are actually running on its servers?
I find this a bit odd and backwards. Why do we choose not to make the deployment code to be robust, but instead to observe the model inference as proven? This moves unobservability layer to the system that collects the proofs. Given the assumption that code is fluid and LLMs are rewriting it often, how can we know it isn't being abused against the observer?
Goes without saying, it's also disappointing that this has nothing to do with making the model inference observable for consumers.
Anthropic here is solving one of their own problems, how to make sure you don't combine other company's models with their stuff for example, and make it sound as if they did it for you.
It makes the inference more expensive and helps controlling the users.
> In both cases the question is the same: how can a company be sure of which weights are actually running on its servers?
> the resulting commitment is good for about 16 million requests, after which the provider has to commit again.
> The client downloads the commitment, ~6.5 MB in all, and then one proof per request. Checking a proof takes 17 seconds on 24 threads (and 21 seconds on a single thread), including the offline phase's check
I know they mentioned $ETH in the technical report, but this is exactly the mechanic for a crypto currency Proof-of-Work which is expensive by design.
When they announced it, I was hoping for a 5% overhead validation that the model that served you is really Fable and not Opus.
The quantization bit caught me out, because we saw the boundary crossing from good-context-size to bad is so sharp it feels like longer context requests are sent to quantized models to make more room for more KV-Cache. This would expose those sorts of tricks people can play to have the same GPU do better on short context benchmarks and gracefully degrade on long context.
At work, we did a bit of digging[1] into exactly why BaseTen, Fireworks and Together serve Inkling weights differently, we suspected the issue was a combination of how you do disaggregated prefill (+streaming) and quantization for longer context tasks.
Interesting, but I'm not sure the property it attests is one we care about that much. Or rather, it's one you'd expect Anthropic not to tell you it needs to prove. I mean it tells you that you're not being stiffed by Anthropic using a cheaper model and that they haven't literally been MiTMd, but that's kind of table stakes. Otherwise it's not telling you much useful. It's not telling you that your data hasn't been exfiltrated, and it's not telling you any useful property of the answer.
What are the tradeoffs of an approach like this vs confidential computing primitives and attesting everything from untampered hardware all the way up to the served request? It seems more performant. Is it simply that they would have to reveal too much?
I've read that providers like those you find on OpenRouter can sometimes route to other models; this would catch that.
But I've also heard, as complete rumour, sometimes some named models perform significantly worse on some hosts and the theory is they're running quantized, without that being published. Would this solve that? The post mentions proving a set of weights, but not how those are represented? What would stop a host using true weights for generating the signature, while running on quantized versions?
> We prove an exact integer version of the published weights, and a proof asserts that each checked token is that model’s greedy choice. A reply from a floating-point implementation, or sampled decoding, could depart at some position...
I think it would catch that, at least for open weights. From the repo docs [1] it looks like this type of attestation is even supported for open weight models: you commit with the public weights and publish the seed, so anyone can recompute the commitment from the Hugging Face files and check the provider is running the exact checkpoint and quantization they say they are. That's what they did with their Maverick demo. That said, it only works for greedy decoding and short contexts for now, and it's 70-95x the GPU time, so there may be cheaper ways to do this like TEEs.
Maybe it's possible to tell by looking at the size of that error?
> The integer version is generally expected and empirically tested to be within some small error rate of true floating-point inference generically (even taking into account non-determinism of floating-point computations), though this does not cover adversarially designed prompts.
At first I thought this was work on the far more important problem of proving that LLM output is provable and correct given some source of truth. But it's about the trivial problem of 'proving' which model is being used.
Who will find this useful? I can't really think of a use for it, because if you don't trust the provider of your models, it's game over really - you're already giving them all sorts of private data, you're relying on their service, you have to trust them to some extent and this doesn't really help because unless you are running the service yourself. Seems like more of an internal problem for Anthropic than something that would interest others.
1. it proves your models are running correctly. FP ops tend to deviate in small amounts between hardware providers and tensor ops even more so due to the relatively new and unstable standardisation environment atm. So it proves your HW configuration is delivering outputs correctly. i.e. it's a good test for what is currently a very non-trivial validation environment.
2. it proves your external provider is running the inference they said they are. i.e. it eliminates the concerns of model providers running quantized versions silently behind the scenes since quantization breaks the proof and you need to use a new, updated proof signature for the quantized version.
3. likewise for fine tuned or tampered models. Or outright substitution for another model (ex the accusation by OAI and Anthropic that Kimi and DS and co are periodically forwarding requests to their platforms to harvest responses for knowledge distillation).
All in all it's a nice chain of trust utility if it can evolve out of the toy stage and reach production viability (since rn it's limited to int-only weights and 1k context which is practically useless atm).
It's worth reading past the opening paragraph before asking these kinds of questions:
>When a user sends a request to an AI model, today they have no way of knowing for certain that the model they selected is the one that generated the response. For everyday uses, in which users mostly care about the output and not necessarily which specific model produced it, this is a less pressing concern. But there are several cases where such assurance would be useful. For example, independent evaluators who test AI models before and after release need to know that the model they are evaluating is the model being served. Clients who use a model through a third-party routing service want assurance that the tokens came from the model they paid for. And if a cyberattacker modifies the token stream while pretending to be a given model, a user has no way to detect it from the output alone.
This is from the first paragraph in the Introduction.
I didn't find any of it compelling, perhaps you do. That's what I was asking really, which of these use-cases is valuable for you?
For me if you use a routing service you have to trust it, it sees all your inputs and outputs.
If a cyberattacker can modify the token stream (i.e. man in the middle your traffic), you should be worried about a lot more than which model you're using!
Off the top of my head, it is useful when there are content-level proxies between the client and anthropic's API server, of which there are many these days.
My PhD is on auditability of AI systems, and we are exactly working on the engineering of ZKP in this context. I can confirm through our industry partner (AI Assurance and auditing), that black box auditing is indeed a huge problem for them, and snarks have shown their potential in this context. It's great to see Anthropic confirming our early results.
This obviously an attempt at being able to say a response came from an anthropic model vs say some other model via a model router. As for me, I will keep using my model router, I don't care to be charged more for a name brand. I don't care about the brand at all, it's a commodity.
until your company gets sued for using Dangerous and Shoddy Foreign models and the boss says you can only use the Trustworthy and Premium Aligned labs' models.
i guess the idea here is that if the end user cannot discriminate based on capability above some threshold, the premium that you can charge for more capable models either disappears entirely (if the end user is the principal source of truth on quality) or gets arbitraged down (if the end user is merely an agent of the principal source of truth on quality). to maintain that premium in perpetuity, you just need (a) trusted benchmarking services that can distinguish between premium and generic, and then (b) a way to prove to the end user that a given output is premium.
> A user querying a model today cannot check that the responses came from the specific model that a developer claims is responding
Indeed, I wonder what kind of developer would make users worry about that...? Funny PR stunt coming from a company that was in the news for silently routing ML-related requests to an inferior model.
> Unlike Fable’s other restrictions, such as around cybersecurity and biology, which openly redirect users to a less powerful model with a visible notification, the system card emphasized that this is “not visible to the user.” The model still responds, but uses “interventions to limit Claude’s effectiveness” without telling the user it’s doing so.
I've been writing about the need for provable inference (and training) on and off for a number of years; this is welcome news and a good start, and a significant improvement on prior zk results.
The problems of inference proof for a closed model provider are more difficult than an open model provider -- you can get a probabilistic proof much more cheaply out of four independent inference providers running the same non-batched prompt than this 90-100x compute hit.
Anthropic and others also have a significant problem with these proofs in that they have a lot of incentive internally to have some portions of the pre-prompt secret; I haven't read this newest scheme closely enough to determine if they have a way to hide parts of the prompt with the proof. But if they don't, they are almost certainly not going to want to send these proofs out in the general case.
Last, I'll take this moment to soapbox and say for the researchers out there -- provable inference is nice. Provable training corpus information would be spectacular. It would be very, very, very nice to provide a set of model weights and a proof that they had only been trained on a dataset. To be clear, this is a really hard problem, but it would change fundamentally what levels of trust we could have about a given set of weights.
> As models become capable of improving and deploying themselves ... how can a company be sure of which weights are actually running on its servers?
I find this a bit odd and backwards. Why do we choose not to make the deployment code to be robust, but instead to observe the model inference as proven? This moves unobservability layer to the system that collects the proofs. Given the assumption that code is fluid and LLMs are rewriting it often, how can we know it isn't being abused against the observer?
Goes without saying, it's also disappointing that this has nothing to do with making the model inference observable for consumers.
Anthropic here is solving one of their own problems, how to make sure you don't combine other company's models with their stuff for example, and make it sound as if they did it for you.
It makes the inference more expensive and helps controlling the users.
> In both cases the question is the same: how can a company be sure of which weights are actually running on its servers?
> the resulting commitment is good for about 16 million requests, after which the provider has to commit again.
> The client downloads the commitment, ~6.5 MB in all, and then one proof per request. Checking a proof takes 17 seconds on 24 threads (and 21 seconds on a single thread), including the offline phase's check
I know they mentioned $ETH in the technical report, but this is exactly the mechanic for a crypto currency Proof-of-Work which is expensive by design.
When they announced it, I was hoping for a 5% overhead validation that the model that served you is really Fable and not Opus.
The quantization bit caught me out, because we saw the boundary crossing from good-context-size to bad is so sharp it feels like longer context requests are sent to quantized models to make more room for more KV-Cache. This would expose those sorts of tricks people can play to have the same GPU do better on short context benchmarks and gracefully degrade on long context.
At work, we did a bit of digging[1] into exactly why BaseTen, Fireworks and Together serve Inkling weights differently, we suspected the issue was a combination of how you do disaggregated prefill (+streaming) and quantization for longer context tasks.
[1] - https://isotopes.ai/blog/everyone-is-selling-the-same-model-...
Interesting, but I'm not sure the property it attests is one we care about that much. Or rather, it's one you'd expect Anthropic not to tell you it needs to prove. I mean it tells you that you're not being stiffed by Anthropic using a cheaper model and that they haven't literally been MiTMd, but that's kind of table stakes. Otherwise it's not telling you much useful. It's not telling you that your data hasn't been exfiltrated, and it's not telling you any useful property of the answer.
What are the tradeoffs of an approach like this vs confidential computing primitives and attesting everything from untampered hardware all the way up to the served request? It seems more performant. Is it simply that they would have to reveal too much?
I've read that providers like those you find on OpenRouter can sometimes route to other models; this would catch that.
But I've also heard, as complete rumour, sometimes some named models perform significantly worse on some hosts and the theory is they're running quantized, without that being published. Would this solve that? The post mentions proving a set of weights, but not how those are represented? What would stop a host using true weights for generating the signature, while running on quantized versions?
> We prove an exact integer version of the published weights, and a proof asserts that each checked token is that model’s greedy choice. A reply from a floating-point implementation, or sampled decoding, could depart at some position...
So, no...?
I think it would catch that, at least for open weights. From the repo docs [1] it looks like this type of attestation is even supported for open weight models: you commit with the public weights and publish the seed, so anyone can recompute the commitment from the Hugging Face files and check the provider is running the exact checkpoint and quantization they say they are. That's what they did with their Maverick demo. That said, it only works for greedy decoding and short contexts for now, and it's 70-95x the GPU time, so there may be cheaper ways to do this like TEEs.
[1] https://github.com/anthropics/FlashProve/blob/main/docs/what...
Maybe it's possible to tell by looking at the size of that error?
> The integer version is generally expected and empirically tested to be within some small error rate of true floating-point inference generically (even taking into account non-determinism of floating-point computations), though this does not cover adversarially designed prompts.
At first I thought this was work on the far more important problem of proving that LLM output is provable and correct given some source of truth. But it's about the trivial problem of 'proving' which model is being used.
Who will find this useful? I can't really think of a use for it, because if you don't trust the provider of your models, it's game over really - you're already giving them all sorts of private data, you're relying on their service, you have to trust them to some extent and this doesn't really help because unless you are running the service yourself. Seems like more of an internal problem for Anthropic than something that would interest others.
The main benefits are:
1. it proves your models are running correctly. FP ops tend to deviate in small amounts between hardware providers and tensor ops even more so due to the relatively new and unstable standardisation environment atm. So it proves your HW configuration is delivering outputs correctly. i.e. it's a good test for what is currently a very non-trivial validation environment.
2. it proves your external provider is running the inference they said they are. i.e. it eliminates the concerns of model providers running quantized versions silently behind the scenes since quantization breaks the proof and you need to use a new, updated proof signature for the quantized version.
3. likewise for fine tuned or tampered models. Or outright substitution for another model (ex the accusation by OAI and Anthropic that Kimi and DS and co are periodically forwarding requests to their platforms to harvest responses for knowledge distillation).
All in all it's a nice chain of trust utility if it can evolve out of the toy stage and reach production viability (since rn it's limited to int-only weights and 1k context which is practically useless atm).
>Who will find this useful?
It's worth reading past the opening paragraph before asking these kinds of questions:
>When a user sends a request to an AI model, today they have no way of knowing for certain that the model they selected is the one that generated the response. For everyday uses, in which users mostly care about the output and not necessarily which specific model produced it, this is a less pressing concern. But there are several cases where such assurance would be useful. For example, independent evaluators who test AI models before and after release need to know that the model they are evaluating is the model being served. Clients who use a model through a third-party routing service want assurance that the tokens came from the model they paid for. And if a cyberattacker modifies the token stream while pretending to be a given model, a user has no way to detect it from the output alone.
This is from the first paragraph in the Introduction.
I didn't find any of it compelling, perhaps you do. That's what I was asking really, which of these use-cases is valuable for you?
For me if you use a routing service you have to trust it, it sees all your inputs and outputs.
If a cyberattacker can modify the token stream (i.e. man in the middle your traffic), you should be worried about a lot more than which model you're using!
Off the top of my head, it is useful when there are content-level proxies between the client and anthropic's API server, of which there are many these days.
My PhD is on auditability of AI systems, and we are exactly working on the engineering of ZKP in this context. I can confirm through our industry partner (AI Assurance and auditing), that black box auditing is indeed a huge problem for them, and snarks have shown their potential in this context. It's great to see Anthropic confirming our early results.
I'm blind to the problem being solved here. Can you motivate it? Why do I need more than normal HTTPS on the token API?
So, you got beaten to the punch?
This obviously an attempt at being able to say a response came from an anthropic model vs say some other model via a model router. As for me, I will keep using my model router, I don't care to be charged more for a name brand. I don't care about the brand at all, it's a commodity.
until your company gets sued for using Dangerous and Shoddy Foreign models and the boss says you can only use the Trustworthy and Premium Aligned labs' models.
i guess the idea here is that if the end user cannot discriminate based on capability above some threshold, the premium that you can charge for more capable models either disappears entirely (if the end user is the principal source of truth on quality) or gets arbitraged down (if the end user is merely an agent of the principal source of truth on quality). to maintain that premium in perpetuity, you just need (a) trusted benchmarking services that can distinguish between premium and generic, and then (b) a way to prove to the end user that a given output is premium.
so this solves for (b).
isn't model substitution a vendor accountability thing? and token tampering, is that not infra security?
do either of these need 70-95x the GPU time in "zksnark" proofs?
Are they doing that because they are scared that Moonshot might route queries to Claude?
> A user querying a model today cannot check that the responses came from the specific model that a developer claims is responding
Indeed, I wonder what kind of developer would make users worry about that...? Funny PR stunt coming from a company that was in the news for silently routing ML-related requests to an inferior model.
https://fortune.com/2026/06/10/anthropic-accu-claude-fable-5...
> Unlike Fable’s other restrictions, such as around cybersecurity and biology, which openly redirect users to a less powerful model with a visible notification, the system card emphasized that this is “not visible to the user.” The model still responds, but uses “interventions to limit Claude’s effectiveness” without telling the user it’s doing so.