I work at OpenAI and I can try to explain our logic.
In general, LLM performance is incredibly multidimensional. There's no guarantee of a fixed ordering where the model that is best at math is also the model that's best at jokes. Despite that complexity, much of the variation in performance is explained by just a single variable: model size. In general, models that are bigger will (a) be more performant / reliable, (b) cost more per token, and (c) have better recall of esoteric facts. As you note, our models are ordered:
Luna < Terra < Sol < Astra
Even today, I'd recommend this as a general starting point for picking models. If you care about quality only, pick Astra. If you're a bit price sensitive, pick Sol. If you're very price sensitive, pick Terra or Luna.
Of course, model size doesn't capture all variation. Models that are trained later might have new data or new techniques or new architectures that earlier models lack. You can see this with models from all providers - Sonnet 5.5 outperforms Fable 5.1 in ways, and Gemini 3.8 Flash outperforms 3.1 Pro in ways.
In the case of GPT-6.1 Terra, I'd say Terra is probably better than Astra and Sol at technical communication, similar on vision and computer use (while being faster / cheaper), and a bit worse on most other things. For API developers, it's a nice midrange option if you want something cheaper than Sol but a lot smarter than Luna (especially for vision / computer use / communication). For subscribers, it's not a bombshell release - if you're hitting usage limits, maybe it helps you go a bit further, and if you want clearer communication or faster computer use, it can be a nice option to select.
If you have suggestions on how to name our products better, I'm all ears. Our intention is to make things clear and simple, even if we miss the mark sometimes.
Unfortunately, a lot of the complexity is somewhat irreducible and due to to the actual complexity of the products. Scaling model size is different from scaling reasoning effort which is different from scaling multi-agent count. So if we just swept all of those under a single name of "more IQ", it would make the naming simpler but the experience much worse, as important details would be obfuscated from end users.
Do you then recommend Sol for technical documentation?
What's confusing to me in addition to the naming is that there are no published guidelines I can use, as far as I can see. Put yourself in the user's shoes. On what basis should one pick a model for a given task?
I can understand smaller models being dumber and faster. That aligns well with scenarios where you are willing to compromise; e.g. you're classifying content or asking a model to format text into a table.
But when you want "maximum intelligence, cost and time be damned", there's the implication that bigger models are smarter on a linear scale, and every provider's model release naming is currently conflicting with that expectation.
This isn't just a naming problem, but a communication problem. If I knew that Astra was the best at coding but a bit bad at writing, and Sol was almost as smart but better at technical writing, then that's the kind of thing I expect to be documented.
One possible way: A simple matrix with horizontal columns like "writing", "coding", "math", rows are the models, and each box is a number of stars from 1-5.
Instead, this sort of thing becomes lore, shared by word of mouth.
> I'm all ears. Our intention to make things clear and simple, even if we miss the mark sometimes.
Thanks. I think the very response you wrote here is a better form of marketing for the software engineers than what openAI puts out there. When everything is groundbreaking and everything tops charts, nothing is.
I suspect a lot of target demographic (developers) prefer simpler and honest communication about not just the models high points but low points too. With the linked Terra post, if I saw 1 chart that clearly highlighted the usecases for which Terra is _worse_ than higher tier models, I would be more convinced overall.
Everything trying to occupy the top spot in my brain's limited "top model" representation space is not helpful.
- Understanding how effort and class interact (Is a high effort terra above a low effort Sol? Are larger models mostly always better regardless of effort?)
- understanding when (if ever) a new model tends to be better than an older but higher class one. Is an old astra better than a new terra?
If increasing effort and increasing model class produce improved results in a different way, I miss making it explicit in which way. In other words: What kind of issue warrants going up in effort? what kind of issue warrants going up in model class? What kind of issue is unlikely to be solved by either?
The 4-tier naming is honestly as clear as you can get. I have no idea what people are so confused about.
Terra and Luna are dirt cheap and good enough for 50% (or more) of the typical junior dev's workload, and great for routine work. Sol and Astra have so much written about them at this point that it should be obvious what those are good at. The large/gigantic models for things that need full intelligence and slightly more branching thinking.
The obnoxious/confusing thing is that models are released individually rather than as a family.
Luna < Terra < Sol < Astra, fine. But does Terra 6.1 beat Sol 6? Does Fable 5.1 beat Opus 5.5? If I build something around this Terra 6.1 release, then when Luna 6.2 is released in a month, should I swap?
Run your own evals then if you have this much of a need to "correctly" choose. That'll make it clear.
I'll just stick to mostly Sol on low/medium/high and start experimenting with Terra tomorrow onward to see what it does noticeably better than Luna/Deepseek flash.
Any pareto frontier model, especially from the leading labs is guarnateed to be good enough for whatever your expectations are at this point. A quick few day test should be enough for anyone to make their own conclusions.
I understand the confusion, but isn't this kind of similar to asking does the iPhone 14 Max beat the iPhone 16 Mini? At least with phones there are hard specs we can use to compare, but in a lot of cases the answer will be "it depends"
Since internally you probably have a fairly good intuition for what these models do well, what about assembling a benchmark suite (with multiple dimensions) that produces results that show that intuition, but in a 'somewhat objective' way?
That could solve the 'more IQ' problem, because you'd actually have different dimensions (e.g. for 'raw intelligence', technical writing, emotional understanding, etc) that you could use to communicate the differences in models better.
(Of course people would instantly start complaining that the benchmarks numbers are probably fake etc, but it is what it is.)
Internal evals are still running but GPT-6.1 Terra is an AWESOME release for anyone using the API in agentic production applications where cost matters.
Personally, I pretty much exclusively use claude for coding work (sorry!). Our application is running almost 100% on OpenAI API (except for some fallback models). It is the best API on the market. The models are excellent particularly for any application that is cost sensitive.
Appreciate that these things have orthogonal measurements; with that in mind, given that the name signifies size, what does version number signify?
I.e. are all version numbers independent and Model X 7.5 is not necessarily of the same generation or cut off as Model Y 7.5? Or do all of your 6.1 models have something in common - cut off data, generation, baseline technology, approach, architecture, or something else?
Please can I ask about one parameter you didn't mention here? All of your models have both a reasoning effort setting and also a non-pro vs pro setting. I have some intuitive feel (from experimenting) for the difference a larger vs smaller model makes, and why that's different from adjusting reasoning levels, but I have no real feel for how pro vs non-pro is different in behaviour from adjusting effort.
It’s probably just unavoidable. Like when Apple release a new MacBook Air first and it is (in some ways) faster than the old MacBook Pro.
I think the planets thing is a bit confusing though. I hadn’t really thought about it being based on the size of the planet. I assumed it was based on how far away they were with terra therefore being the base one.
It would be interesting to know which model is internally viewed as being best at jokes, I have a personal humor benchmark and so far models will either steal jokes (from comedians like Maria Bamford), or write something very unfunny.
When I was first learning to have agents interact I had a little script to orchestrate a model telling a joke, a critic collaborator helping develop material and an audience emoji rank them.
A favorite…
Prompt: “Tell a joke about mother in laws”
Gemma2 response: “What’s the best kind of mother in law? A dead one.”
> If you have suggestions on how to name our products better, I'm all ears.
> important details would be obfuscated from end users
name each model however you like, but correlate the length of the model name with the true cost-per-token (e.g. Juno is cheap, Constantinopole is pricy)
that would quickly sort the most important detail to me, while giving me a helpful token/pin/id for sharing experiences with others
> If you have suggestions on how to name our products better, I'm all ears.
For some reason, this is just hilarious to me. Idk if you're being serious or if this is passive aggressive but the thought of OAI actually taking advice from a random HNer and basing their naming scheme off that is peak.
This is one of the last bastions of semi-serious discussion on the internet, and it's full of incredibly high level engineering talent.
Where else would you go? I'm honestly scratching my head thinking of someplace that wouldn't run you out on a rail, but that would give serious feedback. Kernel mailing lists are out, maybe there's something like that which I can't remember off the top of my head, but...
Also, nomenclature is incredibly hard. There are only two hard problems in computer science: cache expiration and naming things.
I don't understand what's so confusing about this.
The lineup of models is still the same.
There are four models: luna, terra, sol, and astra.
Pricing increases for each level. This new 6.1 Terra is 50% cheaper than Sol 6.1.
They are lowering prices for newer releases, so yes if you compare Sol 6.1 to 5.6 Terra it won't look right, but why are you doing that? Look at the same release version for comparable numbers.
Each model improves in successive releases but they maintain pricing and capability coherency within a release version.
Yes, Sol 6.1 is supposed to rival Astra for some work, which is what you'd expect from a later versioned release model against a prior release bigger model.
Assuming they release Astra 6.1 at some point, I expect it'll be better than Sol 6.1, at a higher price.
I don't understand what people are asking for here? No reductions in pricing? Different model names on every version release? Not releasing any models for a version number until they're all ready?
Hey! I think it's confusing because there is a mix of three components:
- Model name
- Version number
- Effort level
All 3 of these are a one-dimensional balance of price/speed, and capability (Loose terms). The confusion is in how these overlap, how they're announced/marketed etc. So you go from 1 dimensions to 3; you can visualize it as a box. For a given task, how do you choose where in this box you select? Which regions are your favorite? Can you map each region of this 3D space to specific tasks? Label them with intuitive names?
Given the operations of the model are opaque, can you see how someone (most people?) might be confused?
But what's the ask? This is the cleanest way I see of them doing it.
I see a ton of criticism and not a single person coming up with anything better, unless it's to take away options, like removing effort level.
What about buying Intel CPUs? A car? Electrical components?
If you're not just using this at a consumer level to ask about cooking recipes (and if that's your use case, you really don't care about API pricing, or any of this, and can just use the default), and are using it professionally, then this seems like a low bar of effort to understand what you're buying.
I do grant you the relationship between model and effort level has an unclear relationship, but there are also tradeoffs in most complex technical products that are not always clear; this is exacerbated by the fact that this product is stochastic.
Why are engineers using this as a professional tool expecting this to be bought like a consumer gadget and then blaming the maker?
You are showing a selection from some third-party interface that mixes together models from six different suppliers: Mistral, Kimi, OpenAI, xAI, Anthropic and Google, with a bunch of old obsolete models.
Are you presenting this as evidence that OpenAI's naming scheme is bad?
If you don't want to spend the time to know the difference between all these different suppliers and their models, then why are you using whatever this program is that gives you so many choices, most of which are bad? Why is this OpenAI's fault?
Effort (as it's used in the context, how many "thinking" tokens to burn) is still a free variable. It's not always shown to the end user but you can set it independently for any of these models.
After that you just have model "strength" and version number. But of course strength is only comparable at equal version number and the two axes are almost parallel.
Anthropic also does this, perhaps to a lesser extent.
It's confusing because the models are released one at a time rather than as a family.
Let's say I have some task that Luna 6.1 cannot do, but Terra 6.1 can. So I build my product around that. In a month when Luna 6.2 is released, then I have to figure out whether it's worth to evaluate and swap or just wait for Terra 6.2.
Part of the confusion is that it's not linear capabilities. One model is not strictly "better" than the other. They have orthogonal capabilities.
I think the names are more of a reference to price. Luna is always cheaper than terra which is cheaper than Sol which is cheaper than Astra (for the same model number).
Agree, it's confusing. I hear people discussing which model to use for what tasks and my head starts to spin because there are so many properties to account for and it's generally hard to reason about LLMs.
"Speed, quality, price: pick two". I think I'd rather see a choice like that than the current convolution of $-per-token, sub quota, latency, simulated reasoning level, model size, context size, etc, etc.
If you daily all these models (and have a modest budget, not some >$400 corporate allowance), then it's pretty obvious that Luna is small and fast, and Sol 6.1 is the big one for jobs that need significantly above average intelligence, comparatively to a human engineer. I use both constantly. Astra is really not relevant for the average dev except to do some moonshot task with a vague verification loop.
Sure Sol 6.1 on low/medium can be considered "cheap" considering its abilities, but it's clearly not meant for the same type of tasks as Luna. You just grow an intuition for when something is solvable with a Luna-sized model. I'm guessing Terra is good for the tasks that you'd typically use Sol 6.1-low for, or for Luna type tasks (or other not-novel problem) where you want to reduce the risks of suboptimal decision making. It's exactly what I'd go with for feature additions where the correct solution is already mostly designed/specced-out.
the only confusing bit was dropping terra for a bit,
Luna (Small), Terra (Medium), Sol (Big), Astra (Bigger)
is perfectly clear, sol beats astra sometimes because Astra 6.1 was scrapped. and note that opus 5.5 beats fable 5.1 most of the time too without a skipped model.
But minor credit to Anthropic, at least they've mostly stuck to their four model names (Haiku, Sonnet, Opus, and Fable). You mostly know where a model is MEANT to be positioned from the name alone. OpenAI cannot keep it consistent from release to release.
But, yes, currently Opus Vs. Fable has the same issue as Sol-6.1 Vs. Astra.
Both companies have screwed up their naming royally. Maybe that's intentional - I'm sure it doesn't hurt Anthropic to have people using Fable a bit longer than they need to and racking up a huge bill in the process.
It's like iPhone 17 being faster than iPhone 16 Pro. Because it's newer and benefits from advances in chip architecture, despite being a lower model tier. Sometimes different iPhone models are released at different times.
The way to think of this is that (current) inferencing hardware has natural cut points of efficient scale (assuming some optimal architecture which at this point is almost always an MOE variant), so you always have an incentive to provide 3-4 model sizes (which nicely enough often means diff speed to answer also).
You are also rolling the training of your models by size and the lat est small model can sometimes beat an older model.
For a while we thought you just train the big model first then distill smaller models from it but since we've learnt the importance of on policy RL (basically the way a small model figures out a problem is very diff from a big model so it's best to RL optimize each model distinctly)
If you scale a model down, or switch out attention with linear attention, or introduce KV-compression, etc., you don't really know how it will affect model performance before you've trained the model.
And once you've spent millions training it, you might as well make it available if it's pareto optimal for some tasks. It also gives you more insight seeing how people use it in practice.
It's especially confusing because Sol 6.1 is mostly just Astra but 10x slower at 1/10th the price - unless you pay for it to be as fast as Astra - at which point it's the same speed and price...
It's nice that they have Terra which seems to fill a void that's been needed since Sol 6 focused on cost (at the expense of quality), Sol 6.1 got a huge boost in quality, but became unusably slow unless you paid Astra prices - at which point, why not just use Astra?
You're just not paying attention. If you run copilot in copilot using copilot effort, it performs much better than copilot in copilot using copilot effort. This is clearly explained if you visit copilot.com using copilot and then search for copilot.
Wait I always thought terra was the smaller model and the further away you went in space the bigger the model (luna is further away so it’s better than terra)
Interesting that both "none" and "low" reasoning levels appeared not to spend any reasoning tokens at all, and in fact low ended up using less tokens and costing less than none here.
"medium" did use 496 reasoning tokens, but that's apparently not enough here to return a visible reasoning trace.
It's neat that we can now compare the Terra without reasoning pelican (cost 0.64 cents) with the equivalently priced Luna with reasoning pelicans (Luna xhigh is 0.40 cents, Luna max is 0.91 cents). I think the Terra low one compares favorably to those, though I'd suggest also comparing them on tasks more representative of real work than generating pelican SVGs!
Can anyone give me the compelling reason for Terra and Sonnet to exist?
Virtually every task I give to either is better served by going up a tier and down a reasoning level, or down a tier and up a reasoning level. I simply don't understand their use case. No testing I've done has shown them to have a niche where there isn't a cheaper-with-equal-results or more-performant-at-same-cost option.
I think for the majority of the audience here who use these models inside of coding agent harnesses w/ subsidized monthly subscriptions, there likely isn't one. Depending on how Terra contributes to the rate limits, it may make sense to use it for some subset of a repeated workflow to get more mileage out of sub.
If you pay API prices for inference in your product stack & have specific use-cases w/ custom evals, these smaller models provide an additional cost/speed lever to adjust. Ultimately, I think the point is to compete on cost for use-cases where Sol on the lowest reasoning is overkill.
If you have a fairly straightforward usecase where Terra performs just as well as Sol, but is faster and cheaper, why not use it?
There is a chasm of performance difference between haiku and sonnet ime. Sonnet is cheaper and faster than Opus, so if you're providing a service via api where cost and speed is important, it's a big difference.
1. In Anthropic's benchmarks, Opus's advantage over Sonnet diminishes at high reasoning levels. Sonnet max actually beats Opus max.
2. There are rote or simple tasks where higher reasoning isn't needed and one model vs the other wouldn't even be distinguishable.
3. How many tokens are used during reasoning depends as much on the task as the reasoning level.
4. Sonnet is faster and half the price per token, so Opus will only be cheaper if it can spend fewer tokens. This won't be the case for all tasks. Opus and Sonnet will also consume the same number of tokens to load context, so the relative price comparison depends a lot on where the task is context-heavy or reasoning-heavy.
On the API we use a lot of Luna (cost efficient) and sometimes Terra level models. I think we never use a larger model. These are models that our app runtimes depend on.
On subscription for coding and stuff I am basically on Astra Ultra all day long.
If you use API in agent or your own integration with software, such models make a lot of sense. We use Terra 5.6 for "question to SQL" translation for example. It's good enough and affordable enough. Luna will probably not handle complex queries well while Sol is too expensive.
Sonnet is way faster than Opus when doing everyday tasks, I think on the subscription plans it's a better option when you need something done faster than Opus' long thinking sessions.
i use sonnet to generate answers to help-desk tickets, which it does for significantly cheaper and insignificantly worse than opus.
if you're just talking about use in codex or claude code, then sure, leave it on opus all the time. but for running routine api requests, sonnet is great.
Cheap and affordable. Dispatching proper tasks to these models can save you a huge % of your quota. Especially if you _know_ the task is simple (ie add something here or there, or archive a bunch of files and reformat kinda thing)
Maybe it's my tasks, but in the cases you're describing, I'm usually better off going down to Luna. I specifically don't see the gap where "It's too complex for Luna-high" and "It's not complex enough for Sol-low" exists.
I've personally found that most models completely collapse at sub-medium effort. I generally run most of my stuff at medium/xhigh with heavy core tasks @ max
Similar 75% price cut trajectory as GPT-5.6 Sol -> GPT-6.1 Sol. Overall, Terra is still half the price of Sol.
But like the original release of Terra, it's still a weird middle child between the small-model-that-got-substantially-better (Luna) and the large-model-that-got-significiantly-cheaper (Sol). The TerminalBench 4.0 cost-per-task results are a bit confusing and I'm not sure why you would pick Terra over Sol in that case. Overall this release seems like a flex on Sonnet 5.5: a model which is having its own middle child problems between the godly Opus 5.5 and the surprisingly capable Haiku 5.5.
They're doing its because they recently cut plan usage by about 50%, leaving Anthropic providing about 5x the value on their plans. [0]
The "cost per task" metric is just a weaselly way to sound like they didn't just double the cost for most users, and aren't having their proverbial lunch money stolen. They can no longer claim to provide more tokens, more capable models, or more usage in general. So they made up a new metric where "task" could be defined to mean whatever they need it to mean.
They had to become VERY specific in their wording to present any kind of value proposition that will entice people.
Looks great on the first chart, but terrible on the Terminal Bench 4.0 where it needs xhigh to come close to 6.1 Sol, and costs more for a slightly worse result.
Not necessarily, they may have selected benchmarks where Terra performs relatively well compared to those models. Pretty much every model release these days just cherry-picks & highlights the benchmarks where their latest model performs best. We'll have to wait for the complete benchmark suite to have a better picture.
These "Pareto frontier" performance vs price graphs have become very popular and I find them mostly uninformative. Pricing is subjective. You chose the position on the X axis, and which measures to use on the Y Axis.
They say it has "A significant step up on factuality and coding".
My browser is underlining "factuality" in red. AI hallucinations or hallucinatiality are still a big problem in my experience.
People keep asking OpenAI for a mode that's as good as Opus 5.5, and they release this instead. Interesting strategy. Let's see how it works out for them.
They don't have anything competitive with Opus 5.5 yet. There is a supposed model in the works, "Bel" last time I paid attention to gossip, that is thought to be better, later this year. Allegedly Anthropic caught OpenAI off guard and / or their different training approaches ended up working better than anticipated.
My expectation is Fable 5.5 is next overall improvement, and my bet is OpenAI's "Bel" will be the next competitive one (or, their lack of safety culture will bite them/us in the process).
There’s also a chance they surrender the dev market altogether to Ant and just focus on the “normie” segment with dots, computer use, excel etc. In which case they won’t have anything competitive to opus-5.5 anytime soon for SWE.
This release is interesting for enterprise at API rates.
At least unless the Terminal Bench 4.0 score is more indicative than the other ones, because that one shows it clearly worse than 6.1 Sol for both performance and cost.
Taking the benchmarks (as reported by OpenAI) at face value, why wouldn't they show Opus 5.5 on the OS world benchmark, and show no Anthropic models at all on the "factual error rate" benchmark
Anthropic models are comically overpriced. They also gobble up an obscene # of tokens to get the same result as others. The price delta is literally 5x.
That terminal bench 4.0 result is... bad? Terra XHigh more expensive than Sol Low and still does worse.
It's very likely this is a next checkpoint of GPT-6 "Sol", over-optimised for the web chat and offered to Codex subscribers as a "please go away" offering.
If this was half the price, I'd be very interested, but at 1/5/0.05? Feels like another panic release because they can't get Astra 6.1 out of the door...
I'm very confused about their model market position/naming schema.
First we had:
> Luna (Small), Terra (Medium), Sol (Big).
Which was fairly smooth after you got used to their planets.
But then they added Astra as the "Bigger," which was mostly still self-coherent:
> Luna (Small), Terra (Medium), Sol (Big), Astra (Bigger)
But then things go confusing, they released Sol-6.1 which sometimes is superior to Astra, and discontinued Terra which made it look like this:
> Luna (Small), Sol-6.1 == Astra
But with Sol-6.1 costing the old Terra-6.0 price, while, as I said performing as good as the old "Bigger" model in many non-3D/UI situations.
Now they're re-introduced Terra, as a 6.1 release, making this even more confusing.
Did Microsoft's chief branding officer move over to OpenAI?
I work at OpenAI and I can try to explain our logic.
In general, LLM performance is incredibly multidimensional. There's no guarantee of a fixed ordering where the model that is best at math is also the model that's best at jokes. Despite that complexity, much of the variation in performance is explained by just a single variable: model size. In general, models that are bigger will (a) be more performant / reliable, (b) cost more per token, and (c) have better recall of esoteric facts. As you note, our models are ordered:
Luna < Terra < Sol < Astra
Even today, I'd recommend this as a general starting point for picking models. If you care about quality only, pick Astra. If you're a bit price sensitive, pick Sol. If you're very price sensitive, pick Terra or Luna.
Of course, model size doesn't capture all variation. Models that are trained later might have new data or new techniques or new architectures that earlier models lack. You can see this with models from all providers - Sonnet 5.5 outperforms Fable 5.1 in ways, and Gemini 3.8 Flash outperforms 3.1 Pro in ways.
In the case of GPT-6.1 Terra, I'd say Terra is probably better than Astra and Sol at technical communication, similar on vision and computer use (while being faster / cheaper), and a bit worse on most other things. For API developers, it's a nice midrange option if you want something cheaper than Sol but a lot smarter than Luna (especially for vision / computer use / communication). For subscribers, it's not a bombshell release - if you're hitting usage limits, maybe it helps you go a bit further, and if you want clearer communication or faster computer use, it can be a nice option to select.
If you have suggestions on how to name our products better, I'm all ears. Our intention is to make things clear and simple, even if we miss the mark sometimes.
Unfortunately, a lot of the complexity is somewhat irreducible and due to to the actual complexity of the products. Scaling model size is different from scaling reasoning effort which is different from scaling multi-agent count. So if we just swept all of those under a single name of "more IQ", it would make the naming simpler but the experience much worse, as important details would be obfuscated from end users.
Do you then recommend Sol for technical documentation?
What's confusing to me in addition to the naming is that there are no published guidelines I can use, as far as I can see. Put yourself in the user's shoes. On what basis should one pick a model for a given task?
I can understand smaller models being dumber and faster. That aligns well with scenarios where you are willing to compromise; e.g. you're classifying content or asking a model to format text into a table.
But when you want "maximum intelligence, cost and time be damned", there's the implication that bigger models are smarter on a linear scale, and every provider's model release naming is currently conflicting with that expectation.
This isn't just a naming problem, but a communication problem. If I knew that Astra was the best at coding but a bit bad at writing, and Sol was almost as smart but better at technical writing, then that's the kind of thing I expect to be documented.
One possible way: A simple matrix with horizontal columns like "writing", "coding", "math", rows are the models, and each box is a number of stars from 1-5.
Instead, this sort of thing becomes lore, shared by word of mouth.
You can start with Artificial Analysis benchmarks but, then you need to test in the real tasks anyway.
> I'm all ears. Our intention to make things clear and simple, even if we miss the mark sometimes.
Thanks. I think the very response you wrote here is a better form of marketing for the software engineers than what openAI puts out there. When everything is groundbreaking and everything tops charts, nothing is.
I suspect a lot of target demographic (developers) prefer simpler and honest communication about not just the models high points but low points too. With the linked Terra post, if I saw 1 chart that clearly highlighted the usecases for which Terra is _worse_ than higher tier models, I would be more convinced overall.
Everything trying to occupy the top spot in my brain's limited "top model" representation space is not helpful.
I think the main problem is surfacing two topics:
- Understanding how effort and class interact (Is a high effort terra above a low effort Sol? Are larger models mostly always better regardless of effort?)
- understanding when (if ever) a new model tends to be better than an older but higher class one. Is an old astra better than a new terra?
If increasing effort and increasing model class produce improved results in a different way, I miss making it explicit in which way. In other words: What kind of issue warrants going up in effort? what kind of issue warrants going up in model class? What kind of issue is unlikely to be solved by either?
The 4-tier naming is honestly as clear as you can get. I have no idea what people are so confused about.
Terra and Luna are dirt cheap and good enough for 50% (or more) of the typical junior dev's workload, and great for routine work. Sol and Astra have so much written about them at this point that it should be obvious what those are good at. The large/gigantic models for things that need full intelligence and slightly more branching thinking.
The obnoxious/confusing thing is that models are released individually rather than as a family.
Luna < Terra < Sol < Astra, fine. But does Terra 6.1 beat Sol 6? Does Fable 5.1 beat Opus 5.5? If I build something around this Terra 6.1 release, then when Luna 6.2 is released in a month, should I swap?
Run your own evals then if you have this much of a need to "correctly" choose. That'll make it clear.
I'll just stick to mostly Sol on low/medium/high and start experimenting with Terra tomorrow onward to see what it does noticeably better than Luna/Deepseek flash.
Any pareto frontier model, especially from the leading labs is guarnateed to be good enough for whatever your expectations are at this point. A quick few day test should be enough for anyone to make their own conclusions.
> > But does Terra 6.1 beat Sol 6?
> Run your own evals then if you have this much of a need to "correctly" choose.
No. OpenAI want my money, it's on them to communicate properly.
I understand the confusion, but isn't this kind of similar to asking does the iPhone 14 Max beat the iPhone 16 Mini? At least with phones there are hard specs we can use to compare, but in a lot of cases the answer will be "it depends"
Since internally you probably have a fairly good intuition for what these models do well, what about assembling a benchmark suite (with multiple dimensions) that produces results that show that intuition, but in a 'somewhat objective' way?
That could solve the 'more IQ' problem, because you'd actually have different dimensions (e.g. for 'raw intelligence', technical writing, emotional understanding, etc) that you could use to communicate the differences in models better.
(Of course people would instantly start complaining that the benchmarks numbers are probably fake etc, but it is what it is.)
I think different domain benchmarks already exist and are widely used.
True, but afaik none that an AI company itself uses to say 'this is what we think showcases the individual strengths of our models the best'.
Internal evals are still running but GPT-6.1 Terra is an AWESOME release for anyone using the API in agentic production applications where cost matters.
Personally, I pretty much exclusively use claude for coding work (sorry!). Our application is running almost 100% on OpenAI API (except for some fallback models). It is the best API on the market. The models are excellent particularly for any application that is cost sensitive.
It's not gonna happen but:
> Despite that complexity, much of the variation in performance is explained by just a single variable: model size.
please release model size :p
Appreciate that these things have orthogonal measurements; with that in mind, given that the name signifies size, what does version number signify?
I.e. are all version numbers independent and Model X 7.5 is not necessarily of the same generation or cut off as Model Y 7.5? Or do all of your 6.1 models have something in common - cut off data, generation, baseline technology, approach, architecture, or something else?
Thanks! :)
Please can I ask about one parameter you didn't mention here? All of your models have both a reasoning effort setting and also a non-pro vs pro setting. I have some intuitive feel (from experimenting) for the difference a larger vs smaller model makes, and why that's different from adjusting reasoning levels, but I have no real feel for how pro vs non-pro is different in behaviour from adjusting effort.
Instead of benchkmarks it'd be useful to get a sort of Radar chart across measured capabilities like computer use, communication, coding, etc.
We're probably headed to further specialized agents where a single ordering or even model size won't mean a lot.
This makes a lot of sense.
It’s probably just unavoidable. Like when Apple release a new MacBook Air first and it is (in some ways) faster than the old MacBook Pro.
I think the planets thing is a bit confusing though. I hadn’t really thought about it being based on the size of the planet. I assumed it was based on how far away they were with terra therefore being the base one.
It would be interesting to know which model is internally viewed as being best at jokes, I have a personal humor benchmark and so far models will either steal jokes (from comedians like Maria Bamford), or write something very unfunny.
^_^ Would love to see that.
When I was first learning to have agents interact I had a little script to orchestrate a model telling a joke, a critic collaborator helping develop material and an audience emoji rank them.
A favorite…
Prompt: “Tell a joke about mother in laws”
Gemma2 response: “What’s the best kind of mother in law? A dead one.”
What prevents OAI (and Ant for that matter) from releasing new version for all models at once?
Instead, we got Opus 5.5, Sonnet 5.5 and Haiku 5.5 released weeks apart, and so are Sol 6.1 and Terra 6.1?
> If you have suggestions on how to name our products better, I'm all ears.
> important details would be obfuscated from end users
name each model however you like, but correlate the length of the model name with the true cost-per-token (e.g. Juno is cheap, Constantinopole is pricy)
that would quickly sort the most important detail to me, while giving me a helpful token/pin/id for sharing experiences with others
naming suggestion: small, medium, large, x-large
it's almost like ya'll let the AI name it ;)
> If you have suggestions on how to name our products better, I'm all ears.
For some reason, this is just hilarious to me. Idk if you're being serious or if this is passive aggressive but the thought of OAI actually taking advice from a random HNer and basing their naming scheme off that is peak.
This is one of the last bastions of semi-serious discussion on the internet, and it's full of incredibly high level engineering talent.
Where else would you go? I'm honestly scratching my head thinking of someplace that wouldn't run you out on a rail, but that would give serious feedback. Kernel mailing lists are out, maybe there's something like that which I can't remember off the top of my head, but...
Also, nomenclature is incredibly hard. There are only two hard problems in computer science: cache expiration and naming things.
That’s the issue - this isn’t an engineering problem, it’s a general consumer problem. It’s what focus groups full of regular people are for.
I don't understand what's so confusing about this.
The lineup of models is still the same.
There are four models: luna, terra, sol, and astra.
Pricing increases for each level. This new 6.1 Terra is 50% cheaper than Sol 6.1.
They are lowering prices for newer releases, so yes if you compare Sol 6.1 to 5.6 Terra it won't look right, but why are you doing that? Look at the same release version for comparable numbers.
Each model improves in successive releases but they maintain pricing and capability coherency within a release version.
Yes, Sol 6.1 is supposed to rival Astra for some work, which is what you'd expect from a later versioned release model against a prior release bigger model.
Assuming they release Astra 6.1 at some point, I expect it'll be better than Sol 6.1, at a higher price.
I don't understand what people are asking for here? No reductions in pricing? Different model names on every version release? Not releasing any models for a version number until they're all ready?
Hey! I think it's confusing because there is a mix of three components:
All 3 of these are a one-dimensional balance of price/speed, and capability (Loose terms). The confusion is in how these overlap, how they're announced/marketed etc. So you go from 1 dimensions to 3; you can visualize it as a box. For a given task, how do you choose where in this box you select? Which regions are your favorite? Can you map each region of this 3D space to specific tasks? Label them with intuitive names?Given the operations of the model are opaque, can you see how someone (most people?) might be confused?
But what's the ask? This is the cleanest way I see of them doing it.
I see a ton of criticism and not a single person coming up with anything better, unless it's to take away options, like removing effort level.
What about buying Intel CPUs? A car? Electrical components?
If you're not just using this at a consumer level to ask about cooking recipes (and if that's your use case, you really don't care about API pricing, or any of this, and can just use the default), and are using it professionally, then this seems like a low bar of effort to understand what you're buying.
I do grant you the relationship between model and effort level has an unclear relationship, but there are also tradeoffs in most complex technical products that are not always clear; this is exacerbated by the fact that this product is stochastic.
Why are engineers using this as a professional tool expecting this to be bought like a consumer gadget and then blaming the maker?
These names are not just for engineers.
Why is it ok for engineers to have confusing names and ways to interact with something?
How am I supposed to decide between these options: https://imgur.com/a/bXHWowL
How am I supposed to keep up with the changes every few months?
You are showing a selection from some third-party interface that mixes together models from six different suppliers: Mistral, Kimi, OpenAI, xAI, Anthropic and Google, with a bunch of old obsolete models.
Are you presenting this as evidence that OpenAI's naming scheme is bad?
If you don't want to spend the time to know the difference between all these different suppliers and their models, then why are you using whatever this program is that gives you so many choices, most of which are bad? Why is this OpenAI's fault?
Effort (as it's used in the context, how many "thinking" tokens to burn) is still a free variable. It's not always shown to the end user but you can set it independently for any of these models.
After that you just have model "strength" and version number. But of course strength is only comparable at equal version number and the two axes are almost parallel.
Anthropic also does this, perhaps to a lesser extent.
It's as confusing as you want it to be.
Luna doesn't exist anymore (it's not updated)
Larger models solve harder problems, higher effort makes models work more throughly.
It's confusing because the models are released one at a time rather than as a family.
Let's say I have some task that Luna 6.1 cannot do, but Terra 6.1 can. So I build my product around that. In a month when Luna 6.2 is released, then I have to figure out whether it's worth to evaluate and swap or just wait for Terra 6.2.
Part of the confusion is that it's not linear capabilities. One model is not strictly "better" than the other. They have orthogonal capabilities.
I think the names are more of a reference to price. Luna is always cheaper than terra which is cheaper than Sol which is cheaper than Astra (for the same model number).
> I think the names are more of a reference to price. Luna is always cheaper than terra which is cheaper than Sol which is cheaper than Astra
Per token, yes. But per task it's not even necessarily the case depending on how much thinking is needed.
Agree, it's confusing. I hear people discussing which model to use for what tasks and my head starts to spin because there are so many properties to account for and it's generally hard to reason about LLMs.
"Speed, quality, price: pick two". I think I'd rather see a choice like that than the current convolution of $-per-token, sub quota, latency, simulated reasoning level, model size, context size, etc, etc.
If you daily all these models (and have a modest budget, not some >$400 corporate allowance), then it's pretty obvious that Luna is small and fast, and Sol 6.1 is the big one for jobs that need significantly above average intelligence, comparatively to a human engineer. I use both constantly. Astra is really not relevant for the average dev except to do some moonshot task with a vague verification loop.
Sure Sol 6.1 on low/medium can be considered "cheap" considering its abilities, but it's clearly not meant for the same type of tasks as Luna. You just grow an intuition for when something is solvable with a Luna-sized model. I'm guessing Terra is good for the tasks that you'd typically use Sol 6.1-low for, or for Luna type tasks (or other not-novel problem) where you want to reduce the risks of suboptimal decision making. It's exactly what I'd go with for feature additions where the correct solution is already mostly designed/specced-out.
the only confusing bit was dropping terra for a bit,
Luna (Small), Terra (Medium), Sol (Big), Astra (Bigger)
is perfectly clear, sol beats astra sometimes because Astra 6.1 was scrapped. and note that opus 5.5 beats fable 5.1 most of the time too without a skipped model.
I guess it's the same as Opus 5.5 being in a way better than any Fable.
That's definitely a problem too.
But minor credit to Anthropic, at least they've mostly stuck to their four model names (Haiku, Sonnet, Opus, and Fable). You mostly know where a model is MEANT to be positioned from the name alone. OpenAI cannot keep it consistent from release to release.
But, yes, currently Opus Vs. Fable has the same issue as Sol-6.1 Vs. Astra.
Yeah this isn't intuitive at all.
Both companies have screwed up their naming royally. Maybe that's intentional - I'm sure it doesn't hurt Anthropic to have people using Fable a bit longer than they need to and racking up a huge bill in the process.
It is: Fable 5.5 will be better than Opus 5.5. Opus 6 will be better than Fable 5.5, Fable 6 will be better than Opus 6, etc.
You're comparing different generations, any 5.5 models could be better than any 5.0 model.
Really? Haiku 5.5 is better than Fable 5?
I don't know why this is so hard to understand.
It's like iPhone 17 being faster than iPhone 16 Pro. Because it's newer and benefits from advances in chip architecture, despite being a lower model tier. Sometimes different iPhone models are released at different times.
No idea, i still try to figure out how to the right one per task.
The way to think of this is that (current) inferencing hardware has natural cut points of efficient scale (assuming some optimal architecture which at this point is almost always an MOE variant), so you always have an incentive to provide 3-4 model sizes (which nicely enough often means diff speed to answer also).
You are also rolling the training of your models by size and the lat est small model can sometimes beat an older model.
For a while we thought you just train the big model first then distill smaller models from it but since we've learnt the importance of on policy RL (basically the way a small model figures out a problem is very diff from a big model so it's best to RL optimize each model distinctly)
Name means size which means price
Yes, a more modern smaller model can beat an older bigger model at some tasks, that's what can be confusing.
Training models is still not a precise science.
If you scale a model down, or switch out attention with linear attention, or introduce KV-compression, etc., you don't really know how it will affect model performance before you've trained the model.
And once you've spent millions training it, you might as well make it available if it's pareto optimal for some tasks. It also gives you more insight seeing how people use it in practice.
Flip it around. The number (generation) is the most important. Just like it's often with hardware.
And some variants (e.g., the medium one) being skipped or delayed for a certain generation doesn't really change the overall size branding.
It's especially confusing because Sol 6.1 is mostly just Astra but 10x slower at 1/10th the price - unless you pay for it to be as fast as Astra - at which point it's the same speed and price...
It's nice that they have Terra which seems to fill a void that's been needed since Sol 6 focused on cost (at the expense of quality), Sol 6.1 got a huge boost in quality, but became unusably slow unless you paid Astra prices - at which point, why not just use Astra?
Now at least we have a viable other option again.
Huh? I have not observed Sol 6.1 and Astra 6 to be the same.
> Did Microsoft's chief branding officer move over to OpenAI?
Obviously not, otherwise the four models would be called Copilot, as well as the harness.
You're just not paying attention. If you run copilot in copilot using copilot effort, it performs much better than copilot in copilot using copilot effort. This is clearly explained if you visit copilot.com using copilot and then search for copilot.
“Introducing our first mobile devices: Pocket Copilot Luna for on-the-go inference, and our new tablets, Copilot Sol+ and Copilot Astra Duo.”
Wait I always thought terra was the smaller model and the further away you went in space the bigger the model (luna is further away so it’s better than terra)
Next AGI test, when model companies have consistently good version branding.
You're not alone on this. They have too many models, and too many effort levels, it starts to feel like what to chose.
I suspect 6.1 Astra is the model that kept showing up in the news stories about an internal unreleased model trying to hack the Pentagon or whatever.
also am I the only person who remembers Do Kwon and the Terra/Luna debacle?? how about Gaia/Selene (admittedly harder to non-native english speakers)
Bet on purpose
“Test them all”
Pelicans for GPT-6.1 Terra: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
Interesting that both "none" and "low" reasoning levels appeared not to spend any reasoning tokens at all, and in fact low ended up using less tokens and costing less than none here.
"medium" did use 496 reasoning tokens, but that's apparently not enough here to return a visible reasoning trace.
OpenAI's current frontier model lineup is gpt-6-astra, gpt-6.1-sol, gpt-6.1-terra, gpt-6-luna, so here's a comparison grid of all of their pelicans at different reasoning levels: https://static.simonwillison.net/static/2026/openai-pelicans...
12 seconds for a pelican on low. Pretty good.
What's the point of using it without reasoning? Isn't that what Luna's for?
It's neat that we can now compare the Terra without reasoning pelican (cost 0.64 cents) with the equivalently priced Luna with reasoning pelicans (Luna xhigh is 0.40 cents, Luna max is 0.91 cents). I think the Terra low one compares favorably to those, though I'd suggest also comparing them on tasks more representative of real work than generating pelican SVGs!
a bigger model still has more built-in knowledge to draw from, even without reasoning.
Seems Gemini is way ahead in pelican drawing
Apparently Google made it a point to train their models to output SVG, among other things, so that helps
Do you have an agent scanning the front page waiting to post the pelicans?
I should do but no, it's just a bad HN habit.
Can anyone give me the compelling reason for Terra and Sonnet to exist?
Virtually every task I give to either is better served by going up a tier and down a reasoning level, or down a tier and up a reasoning level. I simply don't understand their use case. No testing I've done has shown them to have a niche where there isn't a cheaper-with-equal-results or more-performant-at-same-cost option.
I think for the majority of the audience here who use these models inside of coding agent harnesses w/ subsidized monthly subscriptions, there likely isn't one. Depending on how Terra contributes to the rate limits, it may make sense to use it for some subset of a repeated workflow to get more mileage out of sub.
If you pay API prices for inference in your product stack & have specific use-cases w/ custom evals, these smaller models provide an additional cost/speed lever to adjust. Ultimately, I think the point is to compete on cost for use-cases where Sol on the lowest reasoning is overkill.
If you have a fairly straightforward usecase where Terra performs just as well as Sol, but is faster and cheaper, why not use it?
Idk… I’ve written 100k sloc in ~2 weeks on the Claude £20 subscription with sonnet medium alone. Seems pretty good to me.
There is a chasm of performance difference between haiku and sonnet ime. Sonnet is cheaper and faster than Opus, so if you're providing a service via api where cost and speed is important, it's a big difference.
Except Sonnet is slower, more expensive, and worse at reasoning levels above ~medium. Opus on low is faster, cheaper, and better at doing things.
1. In Anthropic's benchmarks, Opus's advantage over Sonnet diminishes at high reasoning levels. Sonnet max actually beats Opus max.
2. There are rote or simple tasks where higher reasoning isn't needed and one model vs the other wouldn't even be distinguishable.
3. How many tokens are used during reasoning depends as much on the task as the reasoning level.
4. Sonnet is faster and half the price per token, so Opus will only be cheaper if it can spend fewer tokens. This won't be the case for all tasks. Opus and Sonnet will also consume the same number of tokens to load context, so the relative price comparison depends a lot on where the task is context-heavy or reasoning-heavy.
> Sonnet max actually beats Opus max.
What are you smoking and can I have some?
On the API we use a lot of Luna (cost efficient) and sometimes Terra level models. I think we never use a larger model. These are models that our app runtimes depend on.
On subscription for coding and stuff I am basically on Astra Ultra all day long.
If you trust these benchmarks, I'd argue that there's almost never a reason to use anything other than Terra 6.1 xhigh.
If you use API in agent or your own integration with software, such models make a lot of sense. We use Terra 5.6 for "question to SQL" translation for example. It's good enough and affordable enough. Luna will probably not handle complex queries well while Sol is too expensive.
Sonnet is way faster than Opus when doing everyday tasks, I think on the subscription plans it's a better option when you need something done faster than Opus' long thinking sessions.
Sometimes this tier is faster, and getting a good enough result in a few seconds instead of a better result in ~15-30 seconds is a meaningful result.
I think it's also a fake lever to give to finops / it teams to try and keep costs down.
> Can anyone give me the compelling reason for Terra and Sonnet to exist?
To justify the pricing of Sol and Astra. Old business trick.
This is the only reason that makes sense to me. Thank you
i use sonnet to generate answers to help-desk tickets, which it does for significantly cheaper and insignificantly worse than opus.
if you're just talking about use in codex or claude code, then sure, leave it on opus all the time. but for running routine api requests, sonnet is great.
Cheap and affordable. Dispatching proper tasks to these models can save you a huge % of your quota. Especially if you _know_ the task is simple (ie add something here or there, or archive a bunch of files and reformat kinda thing)
Maybe it's my tasks, but in the cases you're describing, I'm usually better off going down to Luna. I specifically don't see the gap where "It's too complex for Luna-high" and "It's not complex enough for Sol-low" exists.
I've personally found that most models completely collapse at sub-medium effort. I generally run most of my stuff at medium/xhigh with heavy core tasks @ max
Similar 75% price cut trajectory as GPT-5.6 Sol -> GPT-6.1 Sol. Overall, Terra is still half the price of Sol.
But like the original release of Terra, it's still a weird middle child between the small-model-that-got-substantially-better (Luna) and the large-model-that-got-significiantly-cheaper (Sol). The TerminalBench 4.0 cost-per-task results are a bit confusing and I'm not sure why you would pick Terra over Sol in that case. Overall this release seems like a flex on Sonnet 5.5: a model which is having its own middle child problems between the godly Opus 5.5 and the surprisingly capable Haiku 5.5.
I don't think software devs are the audience for 6.1 Terra, but instead computer work, automation, and technical writing.
Well put.
Am I reading the Agents’ Last Exam v1 graph correctly where it's saying GPT Luna Medium performs better than High, but XHigh is better than Medium?
I rather enjoy OpenAI's focus on cost-per-task.
I'm optimistic about them tackling frontier intelligence next. Astra 6.1 please?
They're doing its because they recently cut plan usage by about 50%, leaving Anthropic providing about 5x the value on their plans. [0]
The "cost per task" metric is just a weaselly way to sound like they didn't just double the cost for most users, and aren't having their proverbial lunch money stolen. They can no longer claim to provide more tokens, more capable models, or more usage in general. So they made up a new metric where "task" could be defined to mean whatever they need it to mean.
They had to become VERY specific in their wording to present any kind of value proposition that will entice people.
[0] https://newsletter.semianalysis.com/p/anthropic-subscription...
(For some reason they say "Cache read" instead of "Cached input" for 6.1 Terra)
6.1 Astra and 6.1 Luna must be coming up soon. Will they reduce their prices further?
I guess 5x/95% is where we're landing with the 6.1 family, that's nice.
I am wondering if there is a site with a perf/efficiency chart of the models over time? I am wondering is the pace still speeding up?
I'll wait for the runebench results!
On AA intelligence index, this fills in the cost per task frontier nicely between 6 Luna and 6.1 sol. So when 6.1 Luna inevitably releases...
The OAI model tango seems to not have an end in sight, for better or worse. Give me more API toys!
Wow. If those charts are true, this effectively matches GPT-6-Astra at a fraction of the cost, and is neck-and-neck with 6.1-Sol.
Looks great on the first chart, but terrible on the Terminal Bench 4.0 where it needs xhigh to come close to 6.1 Sol, and costs more for a slightly worse result.
I would guess only on the nitpicked benchmarks?
The pelicans place it nicely between Sol and Luna in terms of both pricing and results.
pelican is also nitpicked (and probably the most useless)
Yes, it may be dumb, but it isn't useless because it is both a) is a benchmark and b) is not picked by openai
Not necessarily, they may have selected benchmarks where Terra performs relatively well compared to those models. Pretty much every model release these days just cherry-picks & highlights the benchmarks where their latest model performs best. We'll have to wait for the complete benchmark suite to have a better picture.
These "Pareto frontier" performance vs price graphs have become very popular and I find them mostly uninformative. Pricing is subjective. You chose the position on the X axis, and which measures to use on the Y Axis.
Sell it to me. Why would I ever give money for this?
You are paying API prices, and 6.1 Terra gets you 90% of what 6.1 Sol gets you at half the price.
My understanding is API prices are more expensive then monthly plan, is this true?
My plan for OpenAI recently expired and now I’m thinking to test Anthropic ecosystem. My primary use case is programming only.
Should I go with API use or subscription? For coding harness, I jump between codex, OpenRouter, pi, whatever coding harness is trending on that day
anthropic subscription does not allow 3rd party harnesses i believe
They say it has "A significant step up on factuality and coding". My browser is underlining "factuality" in red. AI hallucinations or hallucinatiality are still a big problem in my experience.
First known use of factuality was in 1859. It's a "real" word. You can look it up.
i'm not sure anyone cares about Terra though.
Previous Terra model was so good it was discontinued.
OpenAi needs a model to compete with Opus 5.5, and that's not Terra 6.1.
Underwhelming
Is it stronger and cheaper than DeepSeek V4.1 Flash 1017?
If not, yawn...
Do people use deepseek for real world development or just for building experimental large projects?
It's only a little better than Deepseek with Max effort in benchmarks. Doesn't bode too well for 6.1 Luna.
"better writing"
let's see if it can pass the Pangram test. Many services purport to fool it, but fail.
Fooling Pangram and writing well are not the same axes, even if there’s some correlation.
Everyone defaults to checking pangram when AI is susepcted, so it's still an important test.
Because Pangram is snake oil.
The benchmarks are already saturated and the curves for Terra stick to the axes.
People keep asking OpenAI for a mode that's as good as Opus 5.5, and they release this instead. Interesting strategy. Let's see how it works out for them.
They don't have anything competitive with Opus 5.5 yet. There is a supposed model in the works, "Bel" last time I paid attention to gossip, that is thought to be better, later this year. Allegedly Anthropic caught OpenAI off guard and / or their different training approaches ended up working better than anticipated.
My expectation is Fable 5.5 is next overall improvement, and my bet is OpenAI's "Bel" will be the next competitive one (or, their lack of safety culture will bite them/us in the process).
There’s also a chance they surrender the dev market altogether to Ant and just focus on the “normie” segment with dots, computer use, excel etc. In which case they won’t have anything competitive to opus-5.5 anytime soon for SWE.
This release is interesting for enterprise at API rates.
At least unless the Terminal Bench 4.0 score is more indicative than the other ones, because that one shows it clearly worse than 6.1 Sol for both performance and cost.
Taking the benchmarks in the article at face value, this is ~on par with Opus 5.5 for ~1/4 the price!
Taking the benchmarks (as reported by OpenAI) at face value, why wouldn't they show Opus 5.5 on the OS world benchmark, and show no Anthropic models at all on the "factual error rate" benchmark
Which is why I don't take the benchmarks at face value...
So you have decided the model is bad without having used it yet?
If they had a model on par with Opus 5.5 there would be no need to undercut it by that much, that's probably the logic.
Anthropic models are comically overpriced. They also gobble up an obscene # of tokens to get the same result as others. The price delta is literally 5x.
...no it's not? At max effort, it scores 50% on terminal bench vs. Opus 5.5's 70%.
ALE (top benchmark in post) is 60% for Terra vs 63% for Opus - I guess we'll have to wait for AA Index!
48 for GPT-6.1 Terra Max vs 58 for Opus 5.5 Max on AA Index
I hate Opus 5.5, constant overreach and over-engineering for any project
That's wild to say when comparing to the OpenAI family of models. I've never seen so many tests that validate tests that validate tests.
You’re holding it wrong.
That terminal bench 4.0 result is... bad? Terra XHigh more expensive than Sol Low and still does worse.
It's very likely this is a next checkpoint of GPT-6 "Sol", over-optimised for the web chat and offered to Codex subscribers as a "please go away" offering.
If this was half the price, I'd be very interested, but at 1/5/0.05? Feels like another panic release because they can't get Astra 6.1 out of the door...
If they had any courage, they would release 6.1 Astra, but they don't.
The next model will likely be Nova