Every single model I've tested that is supposed to be anywhere near Opus, wasn't there. There is some magic sauce in Anthropic models that these benchmarks aren't demonstrating.
I know the log scale of that chart is valuable when surveying the full breadth of possibilities, but I would encourage people to make decisions between nearby models based on linear scaling. Linear cost is often more useful for making the actual decision.
Would you please elaborate on "really decent"? Really decent for what use exactly? I want to love it (and use it as a cheaper research model in my harness), but I really cannot get any real development done with it? It stumbles and fumbles and over-complicates, where astra just plows ahead (often in less overall time) with excellent solutions.
I really want to love this model, particularly since it's good to have an open weight model on my team so to speak, but I think I might be holding it wrong.
Every single model I've tested that is supposed to be anywhere near Opus, wasn't there. There is some magic sauce in Anthropic models that these benchmarks aren't demonstrating.
I know the log scale of that chart is valuable when surveying the full breadth of possibilities, but I would encourage people to make decisions between nearby models based on linear scaling. Linear cost is often more useful for making the actual decision.
discussion being had on https://news.ycombinator.com/item?id=50212065
Looking at those charts makes me appreciate DeepSeek 4.1 flash even more. It’s just so cheap and fast while being a really decent model
Would you please elaborate on "really decent"? Really decent for what use exactly? I want to love it (and use it as a cheaper research model in my harness), but I really cannot get any real development done with it? It stumbles and fumbles and over-complicates, where astra just plows ahead (often in less overall time) with excellent solutions.
I really want to love this model, particularly since it's good to have an open weight model on my team so to speak, but I think I might be holding it wrong.
What are you using it for?
Coding. What were you using last year? It's better than that.