28 points | by mbauman 2 hours ago
5 comments
It's kind of interesting this is already out of data as it's missing Kimi 3 and Opus 5.
Please forgive my naivety, but are world models (once they are in a consumer-ready form) expected to outperform any currently existing LLM on these sorts of tasks (i.e. of the physical world)?
I'd expect google to do well here, since they were historically strong at multimodal and physics.
Nice! Is is missing Codex in the agent harnesses comparison IMO.
Yet another "benchmark to promote their own harness"
It's kind of interesting this is already out of data as it's missing Kimi 3 and Opus 5.
Please forgive my naivety, but are world models (once they are in a consumer-ready form) expected to outperform any currently existing LLM on these sorts of tasks (i.e. of the physical world)?
I'd expect google to do well here, since they were historically strong at multimodal and physics.
Nice! Is is missing Codex in the agent harnesses comparison IMO.
Yet another "benchmark to promote their own harness"