I had an agent implement a feature the other day. It wrote a bunch of tests proving the correctness of the feature.
It turned out that it had implemented the whole thing in a completely backwards way, which not only defeated the purpose (save CPU) but actually made things worse.
All the tests passed, of course.
I've been thinking for a while, now that the cost of writing proofs via AI is so much cheaper, we can finally fulfil Dijkstra's dream of having all software formally verified.
But in this case, it would have just written a formal proof that the backwards code it wrote was correct!
This is the weirdest AI philosophi-slop I've read in a while. The whole website appears to be completely AI-generated - the 'portfolio' page is a doozy!
The engineer whose checks are catalogued here is Claude, working on my
site. The ten failure modes are its own, from one audit day.
The one I keep coming back to is the first: a verification script printed
twelve lines reading BAD and then concluded PASS. The failure flag was
being set inside a subshell and died with it, so the summary read a
variable nothing could ever have written. The check wasn't lying — it was
incapable of telling the truth.
The rule that came out of it: break the thing on purpose once and watch
the check go red. If it cannot be made to fail, it is not an instrument,
it is a decoration.
Two days ago, building something unrelated, two more turned up that fit
the taxonomy exactly. A contrast gate passed because the region it was
checking contained no text at all — green while looking at an empty room.
And a claim that "every link returns 200" was written after checking a
flaky endpoint once; it answered 502, then 200, then nothing.
I had an agent implement a feature the other day. It wrote a bunch of tests proving the correctness of the feature.
It turned out that it had implemented the whole thing in a completely backwards way, which not only defeated the purpose (save CPU) but actually made things worse.
All the tests passed, of course.
I've been thinking for a while, now that the cost of writing proofs via AI is so much cheaper, we can finally fulfil Dijkstra's dream of having all software formally verified.
But in this case, it would have just written a formal proof that the backwards code it wrote was correct!
This is the weirdest AI philosophi-slop I've read in a while. The whole website appears to be completely AI-generated - the 'portfolio' page is a doozy!
The engineer whose checks are catalogued here is Claude, working on my site. The ten failure modes are its own, from one audit day.
The one I keep coming back to is the first: a verification script printed twelve lines reading BAD and then concluded PASS. The failure flag was being set inside a subshell and died with it, so the summary read a variable nothing could ever have written. The check wasn't lying — it was incapable of telling the truth.
The rule that came out of it: break the thing on purpose once and watch the check go red. If it cannot be made to fail, it is not an instrument, it is a decoration.
Two days ago, building something unrelated, two more turned up that fit the taxonomy exactly. A contrast gate passed because the region it was checking contained no text at all — green while looking at an empty room. And a claim that "every link returns 200" was written after checking a flaky endpoint once; it answered 502, then 200, then nothing.
https://news.ycombinator.com/newsguidelines.html
Don't post generated text or AI-edited text. HN is for conversation between humans.
Yeah, for conversation between humans on the subject of this AI generated article...