The readme has a lot of words but still does not clearly explain what this does or what you'd use it for.
The best I can gather is that this captures telemetry while you're running some computationally-expensive job, and checks that the resource usage roughly matches what it's supposed to be. But what does "Tamper-evident" mean? I assume you sign with your own private key? What exactly prevents the data from being faked?
> "A run of committed code, on attested hardware at time T, reading a model with hash m and a dataset with hash d, produced score S.” If d is a public benchmark, they also confirm it is the real benchmark, without you exposing it. The score is also cross-checked by the compute-cost certificate, which confirms the evaluation consumed the compute a real forward pass over the data requires.
They should really put that front and center. Still not a complete enough description, but something at least.
It didn't sound very convincing. It doesn't really prove anything other than the author signed the report. Unless the auditing is running from a secure enclave/core that the user has no access to, it can't do what is claimed on the tin.
There alternatives to a secure enclave like building in a risc0 VM that produces a mathematical proof of each computational step that is very expensive to generate and very cheap to verify. THAT is proof.
But this seems to just be collecting evidence and signing it?
The readme has a lot of words but still does not clearly explain what this does or what you'd use it for.
The best I can gather is that this captures telemetry while you're running some computationally-expensive job, and checks that the resource usage roughly matches what it's supposed to be. But what does "Tamper-evident" mean? I assume you sign with your own private key? What exactly prevents the data from being faked?
Found in the docs:
https://kveritas.org/docs/benchmarks - What a verifier learns:
> "A run of committed code, on attested hardware at time T, reading a model with hash m and a dataset with hash d, produced score S.” If d is a public benchmark, they also confirm it is the real benchmark, without you exposing it. The score is also cross-checked by the compute-cost certificate, which confirms the evaluation consumed the compute a real forward pass over the data requires.
They should really put that front and center. Still not a complete enough description, but something at least.
It didn't sound very convincing. It doesn't really prove anything other than the author signed the report. Unless the auditing is running from a secure enclave/core that the user has no access to, it can't do what is claimed on the tin.
There alternatives to a secure enclave like building in a risc0 VM that produces a mathematical proof of each computational step that is very expensive to generate and very cheap to verify. THAT is proof.
But this seems to just be collecting evidence and signing it?