Google somehow managed to snatch defeat from the jaws of success with their AI products.
They literally forced me and my company out of Antigravity by phasing out AI Ultra subscription without any proper product follow-up. Antigravity IDE cannot even have poweruser subscriptions now from Google Workspace an Gemini Enterprise Agent Platform cannot be attached to Antigravity IDE.
Gemini Enterprise Agent Platform has an incredibly abysmal setup process, and if I want to limit spending per-user I have to create projects per user. The fact that you cannot activate Anthropic models on it if the billing still has free credits is almost a joke.
I was a big proponent of Google and Gemini, but they left us reeling with their abrupt product decisions. Forced us to buy $200 subscriptions directly from Anthropic/OpenAI.
Also a big proponent of Google and Gemini, but their stubbornness in artificially splitting their consumer and enterprise products is extremely annoying. It's pretty weird that I have access to more powerful tools when using my personal Google account compared to my corporate Google Workspace account.
It's literally the meme of the MS org chart pointing guns at each other.
The GCP team wants their slice, the other team wants some otjer slice, and so on. Everyone wants some crap for their promotion package.
It's no wonder Meta has shit the bed even worse.
It's also why Google still releases actually decent, useful models despite the product being such a hilarious mess. A lot of the time Gemini models have actually been better as production LLMs as part of LLM-based production applications than OpenAI and Anthropic models when it comes to the complete cost:quality:latency:adherence picture. And they still are. We have products in prod that use Gemini because they're better than any other model at the specific task. But we wouldn't dare use it for anything coding related, or even just as productivity tool to rely on, because as a consumer product it's a joke.
> The GCP team wants their slice, the other team wants some otjer slice, and so on. Everyone wants some crap for their promotion package.
I got a Google One plan for Gemini, but it came bundled with YT Premium lite, and that somehow made it impossible to renew YT Premium for 30 days. I suspect different teams stealing customers from each other.
Google also gave away 1 year Gemini plans with Pixel phones that either did not work at all for existing Google One users or messed up subscriptions by downgrading your account to worse plan, or making your existing paid time shorter if you been on cheaper plan or recently changee countries. Etc.
Like when you try to give Google money they try to squeeze you as much as possible.
At the same time you can get 5 time more limits for free just by registering 10 free Google accounts.
As someone who's been using Workspace as a personal email account for over a decade this has been such a struggle forever. Just lots of odd limitations to feature sets all over the place.
When they swapped Google Assistant for Gemini as the default voice provider in Android Auto it was so annoying. My wife's non-work space account can get Gemini to do the normal things like play music and what not, but my Workspace one can't do much of anything at all. I can talk about nearly any random topic with it, but getting it to change the playlist, nah, can't help you there.
It's no surprise to me to see them fumble actually supporting a lot of the consumer features of Gemini into Workspace.
I'm in the same situation. But I was shocked discovering it goes both ways: many new Gemini functionalities are only accessible using a consumer account instead of a Workspace account. Also, Gemini is now the only major AI assistant with no support for MCP connectors. Instead of adding this to the core product, like ChatGPT and Claude did, somebody at Google decided that it was smarter to add this fundamental feature to a new product instead: for enterprises this is Gemini Enterprise (which is a product completely different from the Gemini app); for consumers this the new Gemini Spark agent (meaning that you can use MCP within Spark but not within a "non-agentic" chat)... It's clear to me this a symptom of Google shipping their org chart, which is a disaster from a product perspective.
This drove me bonkers. You can enable play music (etc) in Android Auto for workspace accounts by enabling apps in Gemini. From memory (looking at the settings now, not 100% sure of the magic steps required), but go to admin.google.com, go to 'generative ai', 'gemini app', and 'apps settings', then turn on 'other Google apps'. This lets you play music (and other things) in Android Auto.
Also have a workspace as a personal email and ended up getting a personal gmail just to try out the subscriptions before I gave up.
I have multiple anthropic and OpenAI max plans. For Gemini I just use my Cursor $200 a month plan (which also gives me the ability to try grok, conductor, etc)
It's absolute insanity. They have all the resources to have been able to lead from the front with this new technology. They have the products and users to integrate this technology into people's already existing lives. But they keep fumbling.
They're not even benchmarking against other models now, just against themselves - which tells you everything you need to know.
There is absolutely no loyalty when it comes to coding. Nothing could be more common than people threating to jump ship whenever another frontier or open source model comes.
Google is clearly able to keep growing their free and consumer and small business use cases. Unlike corporate coding, we actually have evidence that solo and small businesses can actually see productivity gains.
Anthropic and OpenAI need to stay dancing like mad, because it's their source revenue which underpins their investments.
Why does Google need to shove something at the top at the same desperate cadence? Other than "recursive self improvement leads to AGI" it seems perfectly fine if they push out something dramatically better every year and half.
> Google somehow managed to snatch defeat from the jaws of success
This is still very early days. Who is "on top" has flipped back and forth many times already. The next frontier model release (from whomever) will change things again.
I don't think it's that early tbh, agentic coding has ~90% adoption in the US.
Claude Code has largely won individual developer mindshare and has been on top ever since it came out. The benchmarks change, but almost nobody opts to use anything other than Claude IME when I ask them. Enterprise is more competitive since they care about costs and other things, but developers leaning towards Claude puts a thumb on the scales there.
The product doesn't have much lock in, so it is possible to dislodge Claude, and Anthropic could (and some may argue is likely to) just shoot themselves in the foot again and again and again, but Google has never been particularly good at enterprise sales, and they have never actually been at the frontier of intelligence.
I think Google's incentives have mostly about building models for their products, which makes them focus more on the cheap end, and while they need that, it feels like the Innovator's Dilemma is biting them here.
I own a lot of Google stock from working there in the past and have been quite happy about their trajectory up until the last 6 months, but I am getting pretty antsy about their AI story these days.
> Claude Code has largely won individual developer mindshare and has been on top ever since it came out.
Claude Code's success is not due to the agent but because the model is considered the best for programming and is very heavily subsidized, compared to pay as you go API prices. Consumers and Enterprise are not really locked in and will go where it makes the most sense.
I think they have almost no loyalty by actual developers.
There is no reason to be loyal....There is no moat.
Basically you may choose to drink brand A water bottle, brand B water bottle or tap water. Oh and you might choose the glass water bottle if you use API/Fable.
There’s no reason to be loyal, but I guess it’s a bit like any tool, once you get used to how one works why would you change to another? There is some stickiness with an LLM + harness.
Exactly developers can switch to another coding cli and the learning curve is close to zero. Mindshare without switching costs is just a popsicle in the sun.
Every model has its strengths and weaknesses, being loyal is suboptimal unless you mean being loyal to all of them, which is why cursor would have been well positioned before it got acquired. Now you have to jump through hoops to call Gemini from Claude from codex. Yuck.
Claude code was one of the first agentic code tool and when openai release models similar in performance they didn't do as well in their tools (now codex)
Claude Code (or any other model/infra/harness co-design) is not subsidized in any normal use of that word. It's trough filling (I've written this up in lurid detail so I'm only going to do it again if anyone cares).
Early days or not, Google fucking deleted my IDE and wiped my settings. It took them days to roll out a fix, by which point I had migrated off Antigravity.
I guess? Maybe I'm alone here but I don't feel like Fable is particularly more useful than Sonnet most of the time. I feel like the LLMs are good enough for the majority of uses and the hyper expensive premium ones are way into diminishing returns. At this point with Kimi being as good as it is, if they jack up the price any more I'll just go open source.
I don't think that's true if the reason a company has left the vendor for given model by making it hard to buy. Enterprise IT is enough of a pain in the butt that people will forego the new shiny to avoid the old painful unless it's genuinely better. As you say though, the best frontier model flips regularly, so companies won't go through the hassle of deploying a model if it's proved horrible to do in the past. They'll just skip that model because their current one is fine.
Model quality is only one aspect, the bigger problem is making it work in a fully integrated enterprise platform, and Google has always been lacking when it came to the latter.
At this rate, if Google has a flagship model, you're better off plugging it into a competitor's tooling than hope Google figures out how to use it.
Aren’t the subscriptions extremely subsidized and burning cash for Anthropic and OpenAI? A reasonable explanation is they’re simply abstaining from the war of attrition, especially given cheaper comparable models are breaking the illusion that the “frontier of intelligence” has any kind of per token margin.
This is hotly debated and completely unclear. Let's say Anthropics Opus models cost the same to serve as GLM 5.2. GLM 5.2 is 4.4$/MTok while Opus is 5.6 times more expensive. Assume that GLM 5.2 is served at essentially zero margin. Then Anthropic has >80% margin on API pricing. So even if an average person with a subscription pays only 20% of the API price of their usage, Anthropic makes money on subscriptions.
And the real numbers could be better for Anthropic. It's feasible Opus models are actually cheaper to serve than GLM 5.2 because Anthropic have optimized the hell out of inference.
Sure, but then why wouldn’t I use GLM 5.2 at cost or K3?
I guess that’s the big question, will people pay a big margin long term to use their end products / models or will AI tokens be commoditized by many competing players. For coding if I had to pay API costs I’d switch in a heartbeat, enterprise maybe more reluctant?
Possibly, but aren't the tech giants positioned to win a war of attrition? Then again, they're more likely to sit on that cash and wait for the opportune time to buy a frontier lab.
Google Gemini agy is not allowing you to use your token via your own harness. My own harness is far more efficient than agy. They can take a simple stance - if you exceed your token limit they block you - with the 5 hr limit they are already doing this . so there should be no reason to block you from using your own harness - if you are more efficient - you gain - if you are less you lose .
They are not allowing me to hit their endpoints which agy hits - it's frustrating . i tried to hack it with gemini itself. what i love about gemini is it's so encouraging and ready to help you - even against the agy client : ) .
Even though im so frustrated with this - i still love Gemini for some reason ! Most encouraging model in the world!
> I was a big proponent of Google and Gemini, but they left us reeling with their abrupt product decisions.
Likewise. This seems like a common feel. I have at least spent $4000 and likely a lot more on Gemini API because I really wanted them to win. I gave up.
I am going to ask a very direct question and only because I am curious.
Why do you care? Why would you spend your own money to a multi trillion dollar company so that they win their own "war" against another multi trillion dollar company?
Please don't get me wrong, I know the question can seem a bit negative, I am really just curious.
No, it's a fair question. The answer: I believe(d) in Demis Hassabis's vision of AI
Although, I think saying 'wanted them to win' was not accurate. More like, I stuck with them hoping it will get better, and it did get better in many ways, coding was not one of them.
> they left us reeling with their abrupt product decisions
This is what happens when you put a McKinsey consultant in the role of CEO of an organization where product managers run the asylum instead of engineers.
i've got to wonder how much of this is intentional, and how much of this is just google being their usual terrible selves at anything consumer-product related.
Google's biggest and most important customer for all this AI stuff is google. Do they actually want other customers, or is having other people use their AI just an annoyance at this point, where we use up compute that they'd rather use internally...
It's funny that they triggered the infamous "Code Red" moment in OpenAI when the 3-3.1 models came out. I switched to using them for a lot of single-shot LLM calls because they were fast and cheap. Their only area that they were lacking in was agentic/tool calling.
Needless to say 3.5 was a disappointment. Curious to see 3.6.
> Google somehow managed to snatch defeat from the jaws of success with their AI products.
HN lives in a bubble.
I have German/Italian/Polish clients virtually all use Gemini and NotebookLM. Talking insurance, banking, consulting, legal.
The real world doesn't look at pointless benchmarks on writing react tailwind crap, they are already google suite users, get the tools, test them and adopt them, end of story.
It's going to be like with angular, never mentioned on the net, widely used in the real world.
I have friends and family who use Gemini, but entirely because their Pixel phones came with a year of it for free. No other reason, and they will most likely never pay for it.
They literally forced me and my company out of Antigravity by phasing out AI Ultra subscription without any proper product follow-up
That sounds awful.
For those of us who don't follow the AI hype cycle, what does that have to do with the topic of this thread: Gemini 3.6 Flash, 3.5 Flash-Lite, and 3.5 Flash Cyber?
I wonder how big the Pro model is that Google is using behind the scenes to train these smaller ones.
Going on baseless speculation, the lack of accompanying pro models with these flash releases either means: 1) the model is too big to be economical, 2) google doesn't have the compute to serve the big model, 3) their big model has too many alignment issues to serve to the public.
edit: looks like benchmarks are up on https://artificialanalysis.ai/models/gemini-3-6-flash. It's solidly middle-of-pack. However, if you want to be most fair to flash, look at the intelligence vs time per task and intelligence vs outputspeed benchmarks. This is a very fast model.
edit 2: I use antigravity from time to time and in my experience, 3.5 flash is an underrated model, so long as you know what it's good for. It's very good at frontend (much better than gpt 5.5) and it's fast, so it's a great tool for iteration. I expect 3.6 to be no different.
It's also very possible that they know their big model underperforms chatgpt 5.6 and fable by too much, so they are focusing on what they can get wins in like speed instead.
That and/or the business case isn’t as clear when serving enormous models? You’re constantly stuck in a red queen’s race where your profitability window is increasingly measured in weeks because the Chinese are right behind you.
For small models (which are probably distilled from their big ones) you can serve them economically all the time and not hemorrhage money.
My view then was they are optimising the models for inference ability on their own hardware AND use cases, which is often speed and time to first token.
They've somehow seemed to end up with terrible compute shortages, which again is surprising given how good Google is at infra deployments AND have their own hardware. From rumors out there they are turning down enterprise deals for Gemini because they don't have the compute.
The problem is they're falling further and further behind on frontier class on coding especially, and since I wrote that article it's got even worse with open weights models undercutting them on price AND intelligence.
There was some recent reporting that a July release of the Pro model got pushed back for exactly that reason. Its performance was not good compared to the OpenAI/Anthropic big models. They are having a lot of problems with posttrain.
> focusing on what they can get wins in like speed instead
Speed as a differentiator has always been Google's thing. They (used to?) show the microseconds it took to query & rank web-scale search results. Chrome, notoriously, focused on speed at the expense of resource use. The very many efforts to efficiently speed up Android & its runtime since its inception, and so on...
> their big model underperforms chatgpt 5.6
Possible but TFA claims:
We have started our most ambitious pre-training run yet, for Gemini 4 ...
It would be a shame if they cannot beat Kimi K3 or Qwen3.8 Max, both of which are claimed to be Fable-like. If that is true, it will be [or would be] the first time a major American lab falls behind a Chinese competitor.
I wonder if the broad use of AI overviews on Google search results is having an impact. Maybe the numbers make it more profitable to use their compute on several billion searches a day rather than selling API access.
AI overview is just a summarization of the top 2-3 results. Of course at Google scale that will still need a ton of compute, but the requirement for generating an overview is many orders of magnitude lower than asking the same question in Gemini.
It’s a small multilingual embedding model designed for things like search, RAG, and semantic similarity. It supports a fairly large context window and is designed to run efficiently on a CPU in a GPU starved world.
The interesting part is that it builds on BitNet, using ternary weights of -1, 0, and 1 instead of the usual floating-point weights. That should make indexing and searching large amounts of text much cheaper without giving up too much accuracy.
It seems like there are some credible rumors that Google is actually winning in terms of actually building models that work and don't lose money- between how they're able to price them, the TPU advantage and their capex advantage (being able to raise debt + just having a lot of cash - well I said not lose money... more like not go bankrupt).
From the outside they look like they're behind in terms of frontier models, but I think they might be the best positioned to not go out of business when the bubble pops.
Also look at the fact that they've been able to deploy AI-assisted search at google scale. It must be another order of magnitude larger (at least) than the model deployments for OpenAI and Anthropic.
Of course unless you're inside Google it's impossible to know for sure.
In terms of open models, Gemma 4 beats the pants off everything else to the point that paying for APIs becomes hard to justify. Qwen has the meme-share for coding, but it feels much less well rounded. I have no doubt that Google have both the infrastructure and the expertise to curb stomp everyone else, should they resolve in earnest to do so.
Lest we forget, "Attention is All You Need" came from Google.
Basically, you avoid anything dynamic: model change, tool change, etc it's also important that your system prompt or main prompt doesn't have non-static data like the date/time/place or someone's name (the person you interact with in a chatbot for example). That should be left to tool call or search.
I just put the varying parameters in a trailer prompt and have them change every time. It doesn’t matter because the cache is prefix keyed. You lose caching for the last 20 tokens or so but that’s not a big deal. Moving it to a tool call makes it too slow (needs full roundtrip).
If you’re constructing the prompt you don’t have to jam everything together you can arrange it appropriately.
All of the models, you need to have a consistent input to get the cache hit. So if you are chatting with a document, and change the system prompt, it will be a cache miss, even if the rest of the items are all the same. If you even pass in the document in not the same order as the prompts, it will be a cache miss. Or if you add tool calls or structured outputs, it will be a cache miss. (Since those generally go at the beginning of the prompt call, not at the end.)
Most of the time when reading documents from URLs directly it will never cache. (Need to typically pass in the bytes directly, or use the provider document store index.)
Gemini has a 4096 minimum token size with the 3 version models before even getting a cache hit. OpenAI it is lower (1024), and is automatic, but only happens in increments of 124. Anthropic can also get cache hits at 1024 tokens, but you need to explicit ask for it (and pay extra).
Caching by default typically lives for 5 minutes since the last cache hit across providers. But some of them you can ask for longer. AWS for Anthropic models can be tricky with multiple endpoint routing, so can get cache misses if it happens to route to a different endpoint.
That’s part of why, since Firebase, I’ve tried to never depend on Google products for business, especially not GCP.
Features stay in Beta for ages, whatever that actually means, and released ones get deprecated things fast.
Where some of the competitions treats deprecating entire services as "let’s not put it on your frontpage, put deprecation notices all over the doc, and politely ask new users not to start new project with them".
They know that there's big enterprises that will have a strong preference to work with another big enterprise instead of relying on a younger company. At least that's why I think they believe they can do this sort of thing and get away with it.
its not always that simple. dropping in a new model is trivial, but highly specific workflows may rely on specific _invisible_ aspects of a model. when that model gets deprecated, the workflow needs to be rebuilt/re-tuned to work with a different model.
google's inability or unwillingness to provide stable timelines for model deprecation makes it risky to build complex workflows using their models
You would be surprised how much of a difference the model makes for certain niche tasks.
For my use case, `gemini-3.1-flash-lite` is ~20% higher accuracy than the next best model of comparable cost (considering both proprietary and open-weight alternatives)
Well it is a bit surprising that 3.1 flash-lite could be better than deepseek-v4-pro (cheaper output and way cheaper cache so might cost less for quite a few use cases).
They are not anywhere close according to pretty much every benchmark (even v4-flash is considerably ahead and its way cheaper than flash-lite). Maybe tuning prompts/tools/etc. might be useful?
"Intelligence" being what, math? Coding? Unfortunately there's a billion use cases for LLMs whose performance is not at all captured by the popular benchmarks they're all trying to maxx.
There are plenty of 3rd party providers hosting deepseek models, if you don't want to use the 1st party API. 3rd party providers are generally slightly more expensive, but still quite cheap compared to other models of similar vintage and size.
same here. our production workloads was on Gemini for 2 years. seeing Google unilaterally dropping perfectly fine models and charing you 50x more for worse results is not good.
I felt the same way about openai's text-davinci-002 and code-davinci-002 (gpt-3.5). They were amazing completion models and openai basically dumped them with no equal cost or equal performance replacement. Instead all their models are opaque with no ability to work in completion mode where one actually controls the text input to the model.
These days no company even has completion models where one controls the text input fully. Worthless.
I'm running price-sensitive data extraction workloads on flash 2.5 and its still the king when it comes to accuracy + cost, all the gemini 3 variants perform a bit worse and cost a lot more. Low-key freaking out, ngl
same I just switched to OpenAI after using flash 2.5 lite for almost everything at our company. We spent thousands just to build this workflow now Google says screw off
It's a bit disheartening to see no comparison to other models here - and I'm not sure this pushes the curve anywhere. 3.6 flash is more expensive than GLM 5.2 - but seemingly worse, although this post is really light (lite?) on details.
It seemed for a time that Google had finally gotten the ball rolling, but I'm doubting that more and more as time passes. We'll see what happens with 3.5 pro I suppose.
How does your comparison work? It places Gemini 3.6 Flash Medium above GPT 5.6 Sol High and Fable 5 Medium, which makes me skeptical because that... would be making headlines that I'm not seeing right now.
I have created various questions/tests and put the models through the same tests.
I record whether the answers are correct, and the generation stats (costs, latencies, tokens used, etc.).
I have no idea why the Gemini models do so well.
I have recently added new tests, whose sole purpose was to find some cases on which Gemini 3 Flash fails (I don't like cherry-picking models or tests, but I also find it strange Gemini Flash models leading in accuracy). I made a more complex coding/tool-usage test, that I expected it to fail, it did fail it once locally in my debug tests, but when I finalized the test and ran the entire testing suite for all models, somehow Gemini 3 Flash still got it right...
Gemini models are REALLY intelligent (and they are actually my favorite model to use via the chat app to ask questions), but they somehow fail in real-word coding tasks where they have to modify files, check results, debug, etc.
My tests harness provides a lot of mock data, and limits the number of actions a model can choose from. I am starting to think that maybe the models are not bad, just that the coding harness are not optimized for those type of models, and Google doesn't really provide their own "Codex".
Interesting, well it'd be interesting to check out some individual examples where Gemini beat the others.
Also, would be great if you could add GPT 5.6 Sol XHigh and Fable 5 High as well, just to see if at least those beat Gemini which is currently your #1.
I don't like to divulge tests, but one of them is a chess puzzle.
> would be great if you could add GPT 5.6 Sol XHigh and Fable 5 High as well
I would like too, but I avoided them for several reasons:
1) Cost - this is a hobby project, those models would cost tens of dollars for each benchmark run, multiply this by tens or hundreds of models and ...
2) Time - the high models are already taking a really long answer to respond (5-10minutes per question). I run each question with 3 repeats (run the same test three times), so it would take 30 minutes per test. If I change my tests, methodology, or add a new test, it would take a really long time to run the benchmark. Also, I like having results immediately when a new model is released, now I can post within 30 minutes of a model's release the benchmark results.
3) High reasoning usually does WORSE on most tests - if you look at the leaderboard, it's sometimes counter-intuitive, but models with high or max reasoning usually do worse than medium and low. This is because the questions are quite targeted/direct, and the models overthink the question and miss the solution. Or the long thinking context makes them perform poorly. The generation tasks (SVGs/HTML animation) are usually better with longer reasoning, but short code fixes, trivia questions, puzzles, etc. are answered by low/med reasoning with more accuracy in general
Also, Fable is borderline un-testable, it refuses to answer many questions, so it scores poorly anyway.
Gemini scores 21/22 because it answers all tests, and it does them correctly, consistently. The only failed test is I think because it miscounted the lines in a file, when responding on which line the bug was in a code snippet.
Oh, and I've also added weights to different categories, so Coding and Tool usage categories influence the score more. This done both to better account for how most people are being used, and also to reduce Gemini's dominance in general/domain specific knowledge.
So yes, Gemini models are at the top, even if I actually (not proud of it) tried to make tests that actually favour other coding-focused models.
It’s really surprising. When Apple announced the multi-billion dollar deal with Google to power Apple Intelligence I thought great things were coming. Instead we are getting more and more bad news: delayed Pro models and AI leadership leaving. I wonder if Apple know something the rest of us don’t know or if they are already regretting their decision.
Besides Apple apparently making Siri AI model agnostic, the choice to go with Google was almost certainly for practical reasons. Google is a low-risk established player that already has a long work history with Apple. Google also isn't in an existential battle to establish themselves, Gemini still amounts to just another project at Google. There is tangible non-zero risk that either OAI or Anthropic will be gone in 5 years, or will be forced to leave Apple high and dry to save themselves. There is almost no risk Google will be in either such position. And worst case scenario, Google has incredibly deep pockets should Apple pursue a "refund."
What Apple wants out of Google is Siri that runs at 8gb ram and isn’t a horrible embarrassment that feels like a primitive markov chain. Given how good Gemma 4 is, Google can squeeze some serious performance in small models. Whether they can make bleeding edge models is irrelevant to Apple.
As someone on the Apple beta.. the model is almost completely irrelevant to the experience. Apple has gone and done Apple things by nerfing the experience so completely that almost any model in the past year would be fine. I still reach for ChatGPT/Claude/Grok constantly instead of the AI toy that Apple calls the new Siri.
The one thing I've found google's models to be the best at is proofreading text in non-english languages. Probably because I imagine they have the most training data for it as Google probably has the most complete archive of the internet.
I am growing tired of these pelicans posts every time a new model is published. Feels to me like low effort personal brand promotion. Just sharing my 2 cents.
+1. First thing I look for in a model announcement thread. I actually came across this one an hour ago and was sad there were no pelicans yet.
It's a decent heuristic because the better models generate better pelicans. That's all. Nobody sane is going to make a bet on a model based on a pelican. But it's cool, it's tradition by now, and it's a semblance of a good first impression for new models.
But how else am I supposed to know when we've reached AGI, until I see an absolutely flawless pelican?
All of the pelicans so far have had really weird flaws / quirks so I am always a little interested to see how well these models perform at this task, since I've seen all the past pelicans and have some anchoring.
Seeing a truly flawless pelican would tell me that the model has true visual reasoning capabilities as well as good taste.
At this point, it's kind of a hackernews thing. Simon posts them as a single comment in the relevant thread. It's okay for this place to have a little bit of a sense of community, and you can just ignore the comment.
Disagree. They're a nice tradition, but besides that, they're a useful way of eyeballing improvements. I realise labs are likely to be training for Pelicans - but if they're all training for them, the differences in the results are as indicative as they were before labs trained for them.
The 3.6 Flash pelican is just about the best I've seen.
I agree to rednb that at this point it feels like rather obvious brand building, but also, I agree with you that some value is in it.
It does not feel all that authentic though, and it's good to react allergically to lack of authenticity. Bad for a lot of business models, but good for humanity.
Sorry mate, but you sound jealous in all these replies that the Pelican domain isn't your gig. The below is as labored as nitpicks ever get:
> It does not feel all that authentic though, and it's good to react allergically to lack of authenticity. Bad for a lot of business models, but good for humanity.
I thought the Gemini 3.5 Flash Lite response was quite telling myself. I personally like the Pelican SVG test, to me it is still a charming snapshot of model performance anecdata. No one would argue it's rigorous but I don't think it was ever intended to be.
I get people burning out on the pelican SVG test alongside the rest of the AI burnout, but I guess for myself I'm just choosing to keep enjoying it while I still can.
Not only does it give you a super easy-to-grok understanding of the model quality just by looking at the image, but when you compare tokens and costs (both input and output), you really get a good, simple COST x QUALITY evaluation across models.
Yeah. It's something I can do myself in a couple seconds if I want, also on more varied SVG scenes. If this is going to be a benchmark people turn to I'd like to see more effort put into it than just a one-sentence prompt.
I'll split the difference. When it's a blog post there's usually an interesting observation or two, but if it's totally automated? Maybe just do the ones with a post.
A piece of the frame is missing between pedals and back wheel. The frame of the bike passes through the bird. It also puts a cap on the bird's head, and a fish in it's mouth.
The fish and the cap where always added when I asked an llm to improve it's first attempt.
This continues the trend in LLM progress of better=more stuff
In other good news "the model has been trained to minimize refusals for beneficial uses.".
Otherwise, this news feels like a tiny incremental improvement on Gemini Flash series to make it more efficient with token usage, subagent and cost. Nothing big.
Regarding their benchmark scores on CyberGym, I wonder why they didn't compare their 3.5 Flash Cyber model with Fable 5. I mean they included Mythos and GPT-Cyber, so why not Fable 5 too?
They also mentioned Gemini 3.5 Pro is in testing and its about to become available very soon. Another thing maybe worth discussing is the announcement of pre-training Gemini 4. Sadly, not much technical details to discuss on. Many comments in here seem to mostly be about how Google is behind the others, but honestly, is it really worth the investment to be #1 in Artifical Analysis every week?
3.6 Flash would be a great model at 3.0 flash pricing. At this pricing, its thoroughly trounced by about 10 models on cost/performance including Grok 4.5.
3.5 Flash-ite would be a great model at 2.5 flash-lite pricing, as is, its trounced by many models including Deepseek v4 Flash.
As is, they are thoroughly outclassed for most usecases. I will say the one area where i do see Gemini punching above its weight class is in tasks that are effectively "Google this for me" / knowledge stuff. So it does have a role, and I do use it. So while I think Google is still in a strong position overall, they are really stuck as a tier 2 AI player right now with text models. They are tier 1 in bio, images, and video.
Could also be that they are pricing it at levels where they actually make money. Without seeing the behind the scenes compute cost on all of these its hard to really judge.
That being said with any open model we of course do know the total cost (or estimate)
3.5 Flash was always too expensive for a "flash" model. They marketed it as "near frontier" level, but there are several order-of-magnitude cheaper open models that compete with it.
In my tests, 3.6 Flash is NOT more token efficient, so it actually ends up costing more than 3.5 Flash, even with the output price reduction.
EDIT:
It less less verbose in final output though, but it reasons more.
I assume the optimization comes when you have long-running tasks with many tool calls, and by reasoning more, it reduces the number of tool calls needed.
Pricing often reflects what the vendors (expects) the customer is willing to pay. It seems that Google is still trying to find their niche in the market.
I guess they meant to release Gemini 3.5 Pro shortly after 3.5 Flash, but then Mythos/Fable and later GPT-5.6 came out with higher performance than 3.5 Pro, so the managers decided not to release it.
Really, what's up with Gemini still not supporting connectors/MCPs/plugins/whatever-they're-called-this-month on web? It makes it a non-starter for any kind of serious use.
Is that statement based on token price? More and more it seems that $/token hides as much as it reveals. Token efficiency, tokenizer differences, etc. I'm not saying that you are wrong, I am just saying it is becoming a bit more difficult making statements like this without a bit more research.
It'd be interesting to know how much the Intelligence as a Service angle serves as a value-add in the minds of Google's executives.
You can get decent open-weight models now. That's not difficult. The difficulty is 1) running them and 2) compliance.
My company runs Claude on GCP's Vertex AI solution. We're in the US healthcare IT space, so the models need to be from somewhere that American healthcare agencies and companies have traditionally been okay with sourcing code from - which means the US, Canada, and maybe Europe. The stuff that handles PHI/PII must be in the US. The expense of hosting is more of a PITA than most customers want to go through this early in the technology's lifecycle, and intelligence gains are simply a matter of degree for most business tasks.
In theory, we could find some open-weight model (likely from China) for our development agentic work and host it anywhere you can host AI models. We don't, though, and I think Google, OpenAI/Microsoft, and Anthropic see that as the core of their business.
No word about updating Jules, which is still stuck on 3.1 Pro. I get that it's probably niche but I've really appreciated basically being able to give directions to Jules on my phone, then reviewing and merging a GitHub PR fifteen minutes later. It's been great for getting some progress in on a few personal projects during my commute when I can't exactly pull out my laptop.
Shame. I'm on the $20/mo Gemini Pro plan because the 5tb of cloud storage and the youtube premium lite were good enough perks, and my coding complexity needs were light enough for me to overlook Claude or Codex. But Antigravity is working better than Jules and it's basically giving me a taste of what I'm missing and it's harder to justify not trying out the competitors.
I do not, however I am curious about Jules support. I didn't know if this was a dead project or not. Seemed really interesting but then I didn't see much development/announcements/discussions around it. Last update from their changelog was as you said 3.1-pro support in March.
I have a side business selling custom fingerprint jewelry and I use gemini nano banana to clean up customer submitted fingerprint images. This was a step I used to do by hand at 10 - 15 minutes per image and nano banana is the first model that is able to do the task (it is astonishingly good at it). I can't wait to see what the next nano banana can do, hopefully its released soon.
Google seems to have anorexia when it comes to model intelligence. They have an internal hard constraint on price per token it seems, and they are trying to squeeze out intelligence with limited compute.
I wonder if there is something with their TPU cycles that makes them want to postpone training a new model. My guess is that they have been on the same base model for 6 months and they may have waited for the next gen TPUs to train Gemini 4, which greatly limits how much intelligence they can increase and forces them to do cost efficiency increases.
Google has not changed. Following two facts are like tautologies by now.
1. Their AI efforts are very fundamental research oriented. They are really good at it.
2. Their productization sucks. The end products gets little attention compared to competition. It can be canceled at any time. You should never build anything around Google only APIs, AI or not.
I have just tried to switch to 3.6 instead of 3.5 in antigravity and it seems to constantly spit "critical instruction: STOP CALLING TOOLS NOW. YOU MUST WAIT FOR WAKEUP. ". I think I will switch back to 3.5
Kind of excited about this. 3.5 Flash on Antigravity has surprised me recently on a hobby project. When given opportunity to plan, it can deliver on tasks that would take me a while on my own and generates responses at blazing speeds - compared to what I'm used to at work with Opus 4.8 (granted I don't use Opus 4.8 on my hobby projects so just anecdotal). While with Gemini CLI I would just watch it run in circles and run out of 5h allowance before anything useful is produced (or even approached).
Because AA Coding "Index" consists only of two benchmarks (Terminal-Bench v2.1, SciCode) and generally fails to be meaningfully representative of agentic coding capabilities.
DeepSWE and FrontierCode are more realistic if you read up on what they actually measure. But the most realistic is to try it yourself. Benchmarks can only vaguely represent typical usage, and how you judge the result. Giving the same real task you have to a few models will make you understand them better than chasing benchmarks.
Could be a good tradeoff for the flash model though. 3.5 -> 3.6 is a tiny bit cheaper and maybe faster?
artificialanalysis.ai has it going from 165 tps -> 304 tps. openrouter.ai needs more data but it has it going from ~100 tps -> ~150 tps, though at peak 3.5 has reached 156tps.
Wow - Google does not even bother to show benchmarks of these models compared to the frontier and Chinese labs - only against previous versions. I'm not surprised. Having worked there for years it was amazing just how inwardly looking the company is.
I don't know if it's a rendering error since it looks like your site renders the SVG instead of hosting a static image of it, but the 3.6 macbook looks like an abstract art piece lol, both ff and chrome desktop
I have no skin in this game and this comment will be gray in a few minutes BUT a friendly reminder that these types of threads are astroturfed heavily by competitor labs and any info should be taken with a massive grain of salt.
It is 17% more token-efficient than 3.5 and performs significantly better in coding and tool usage benchmarks.
It is also cheaper than 3.5:
> This enhanced efficiency is also combined with a lower price than 3.5 Flash. At $1.50/1M input tokens and $7.50/1M output tokens, 3.6 Flash reduces the overall cost per agentic task, making agents more cost-effective to build and run.
LLM reception is truly extreme, even worse than AAA game releases.
Ever frontier lab lived it at least once : missing the frontier by a few months triggers extremly negative reactions, then you take back the lead for 2 weeks, and the hype cycle repeats.
Always happy to see new Gemini releases as IMO Antigravity Pro 16.67/mo plan (Annual) is still the best plan available and have been pretty happy with Antigravity IDE.
If it wasn't for Gemini/Antigravity I'd have to go with a Max Claude plan, as it stands now I can get by with just a Claude Pro plan to get Opus when I need it, whilst using Antigravity as my day-to-day workhorse.
Unfortunately Gemini Flash became too expensive to use as a general purpose model (i.e. for AI features in Apps), luckily there are plenty of cheaper Chinese models to fill that gap now.
Don't know why are they even pursuing Gemini. Just download the Kimi, call it Kimini and serve it on your GPU. Maybe then train next architecture based on this!
Gemini 2.5 Flash-Lite has been my go to for cheap document processing at scale (especially with 50% off batch mode), but they are really boiling the frog with pricing increases with each version:
gemini-2.5-flash-lite: $0.10 input / $0.40 output
gemini-3.1-flash-lite: $0.25 input / $1.50 output
gemini-3.5-flash-lite: $0.30 input / $2.50 output (a 6.25x increase over 2.5!)
Now watch them deprecate Gemini 2.5 Flash-Lite in the coming months...
Feels like they released this to ride the wave of press of GPT-5.6, Kimi K3, and Qwen 3.8. Doesn't feel like Google has much substance with this post except a bump in version and tweaked their pricing.
I'm a big fan of the Flash-Lite models. They're exceedingly fast and deliver great outputs for high volume use cases where you need to process requests at scale. Can't wait to try the newer version.
For anyone wanting a faster overview: I ran the Gemini 3.6 Flash and 3.5 series release notes through NotebookLM and generated a short video summary. Link: https://www.youtube.com/watch?v=SUFBhvQ2tY4
IMO Gemini has the best free tier models/app for everyday use. Muse-Spark is perhaps just slightly better, but has none of the connectivity to my GApps (for things like “create a recipe in my Google Docs from this image”).
Plus they are probably running these things on every Google search so saving tokens is a huge win for them.
3.6 Flash scores exactly the same as 3.5 Flash on the Artificial Analysis index. Better on some tasks, worse on others. Mostly within what I'd consider the noise window. Looks pretty much indistinguishable from 3.5 Flash, at least on these benchmarks: https://artificialanalysis.ai/models/gemini-3-6-flash
That's wildly ambitious pricing by Google. You can maybe get away with spicy pricing at the SOTA edge but at the lower tiers everything is a lot more price sensitive.
You need to compare cost per task buddy boy. Cost per token doesn't tell you much when you don't know how many tokens a model will use to accomplish a task
Because they don't have a lot of parameters to store general Wikipedia knowledge. They're small. Use big models that have high parameter capacity to store general information. Or build a harness around the small model that searches a knowledge base/internet.
Use the right tool for the job. It's like asking why a screwdriver isn't good at sawing wood, or calling C a terrible language because it's hard to make CRUD apps with it.
So 3.6 Flash is a somewhat of an admission that Google miscalculated by charging 3-5x for 3.5 Flash what it did for 3.0 Flash (3x input and output costs plus large token inefficiency changes) despite only modest improvements?
3.5 Flash Lite is only a hair cheaper than 3.0 Flash, but I think 3.0 Flash is a massively more capable model?
Proof-of-life release while they figure out how to have a competitive frontier model release. My hunch is they pushed too far in the "omni" model direction, that they made something so ungainly, it wasn't as good for normal tasks.
It does seem like their releases are getting closer together. I get the feeling they realized they were trying to roll out to their entire ecosystem and now they’re focusing more just directly on the AI model itself. I think give it a little time and they’ll start to be one of the competitors too.
They are working on coming up with a better code name. You know, something like ”Fable” or ”Sol”, gotta have one these days. Personally I think they should go with “Mafia”. How cool would that sound? 3.6 Mafia.
Plugged 3.5 Flash Lite into an existing agent harness that was previously using 3.1 Flash Lite and this shit just does not work. It's not following instructions and is not producing the correct tool calls.
the real product is the naming confusion we made along the way. Gemini 3.6 Flash, 3.5 Flash-Lite, 3.5 Flash Cyber — at this point even the model cards need a model to explain them
quite a good model, the speed/price/quality ration is a new golden intersection for me, not sure if its as good as grok 4.5 but quite fast/capable model.
It feels like AI is going to be the end of Google. The post-Schmidt company culture cannot produce consistent, consumer-friendly products that any sane person would want to use consistently.
Especially when Google owns 15% of anthropic and serves them compute. Double especially when your boss (Hassibis) is also an early investor in Anthropic. Hell his NW might be more Anthropic than Google.
Yep, agreed. They still are not releasing anything frontier-class (Gemini Pro) at this point. Feels to me that they keep getting scooped by others (e.g. Kimi 3) and then are retrenching.
Models are expensive and low performance. On top of that they make you jump through hoops to even use these models without being throttled even for the weaker models. The only reason we are using them is credits. As soon as credits run out we are switching immediately.
tl;dr: 3.6 flash is a bit smarter than 3.5 flash, but also a bit more expensive.
My results [0] put Gemini 3.6 Flash at the top.
3.6 Flash high has same $1.5 input price as 3.5 Flash, but output is cheaper from $9.0 to $7.5.
Google said 3.6 Flash is more token efficient, but in my tests it's actually LESS token efficient[1] than 3.5 Flash, so despite the output price reduction, it still costs more.
I remember back when Gemini looked like it was the best model that this comment section was full of confident predictions that Google had "won" and that no one would ever catch up with them again. The most embarassing part is that I kinda believed them.
Why do you care that much? Just say it. You're letting an imaginary score determine if you should express your suggestion for improvement. That's a bit wild.
I think it’s safe to say Google seems a bit out of the top AI competition now. The “cyber” stuff also starts to become laughable with open models providing the full power without the crap Anthropic, Google, OpenAI are trying to frontload on you(I.e you are not allowed to develop/review a login system, pay a special cyber operation team to do it for you). They really deserve to become irrelevant in the future of AI.
Google watches over the last few months a flat out assault on the Pareto curve from American and Chinese companies. Release after release pushing the boundaries of frontier intelligence and price/performance.
And the response from arguably the biggest AI research labs in the world by headcount is Flash 3.6.
What do you do when you are given essentially unlimited resources and still find yourself falling behind?
Gemini 3.5 flash is already a pretty good model. But, unfortunately, the primary way you can interact with it for coding is through Antigravity - which is actively developer hostile.
It doesn't matter how good the model is if you're (mostly) forced to use it in Antigravity - which turns any model into crap.
I use 3.5 flash 10x more than any other model, despite have access to all of them. If I'm going to play a slot machine, I'd rather get the pain over with quickly.
We're almost five years into the whole GenAI thing and we're still relying on these guys to spoonfeed us incremental updates.
It's time for them to start focusing on open-weight models and efficiency. Otherwise there's just a layer of marketing hype and "will it do this?" that has to be cut through for evaluation of each and every release cycle.
Models are getting easier and easier to create. The money, if there's any here, is in the harness the user interfaces with, and the data centers running them.
Google seems to be falling way behind the pack. antigravity cli is pure trash. gpt 3.5 pro is now behind and isn't released yet. GPT 6 and Fable 6 releasing next month. What the hell is going on over there ?
Google was late to coding agents and as-per-usual fucked it up with their crazy project management culture.
Usually Google gets away with it due to inertia, however this time they are paying a heavy price because they missed out on the training data that Anthropic and OpenAI have gathered with claude and codex.
> we have taken an intentional approach to deploying 3.5 Flash Cyber. The model will be exclusively available to governments and trusted partners
Screw your government! US and Israeli governments should get the least access, but of course we all know they'll be the (only) ones to get full unfiltered access.
With all the naysayers on Gemini models I'm curious how many people actually use Gemini regularly?
For me, Gemini models are the most usable. Claude Opus and Mistral always try to turn queries into one-shot enormous commits, which just burns tokens, time and annoys me for something which is still wrong more often than not.
Gemini seems far better at listening to instructions and giving me what I actually want, on top of using far fewer tokens and wasting my time. Fable is the only model that's come close to Gemini Pro for me.
And as this is about Flash, it's exciting, I find Flash can usually get the right answer pretty quickly and without too much nonsense.
I keep saying this and people dont believe me, but I have b2b saas systems with actual agents running around the clock, and the performance/stability of the flash model is higher than most other models.
Meaning, its predictable with tool calls, wont spin off a million tools/do weird behavior, its reasonable. Even sonnet in a real world decision making scenario is not reliable, or will reason so long its incredibly expensive.
The benchmarks arent catching all the value, and most people have never actually ran an ai agent in a real context that matters
Who's most people? What are you talking about? Most people here use agents every day and I wouldn't trust flash or pro to touch any important project of mine because they're both terrible compared to the competition, waste of time every time I give them a chance
not a google fanboy by any stretch... though i've been thrilled with the flash line of models... i exclusively use it on high, and have found it to be a great fit for increasing productivity 10-fold while maintaining quality... sure it can't just go off and one-shot a bunch of work, but at the complexity level i tend to work at, neither can the frontier in a robust way that i can be confident in... sure i have to be in the loop more, but that helps keep me grounded and course-correct earlier before wasting tokens... and when you sufficiently spec out a coding/software problem, and i mean really document all of the critical nuance, it will successfully satisfy the constraints... the quality is rarely acceptable on first-pass, but it forces me to stay connected to the architecture more than i would be if using a frontier model... i've found this to be a happy middle-ground of productivity and awareness...
tested the models on aistudio. despite that the knowledge cut off is march 2026 it still knows nothing about 2025!
you can check by asking "list notable world events in 2025, only list unplanned" on aistudio. or you can ask for Charlie Kirk, it also does not know. I tried it multiple time to ensure that I didn't not get routed to older models!
> but google has search
irrelevant, without deeper knowledge about cutting edge technologies or latest libraries, all of it suggestions are crap. even you ask it to search it will still use outdated keyword thus only getting outdated information.
At this point, I think google should consider becoming a hyper scaler for anthropic and open ai, and I predict that that is exactly what they do. The model is no longer the most valuable part of the stack.
Google desperately needs to make some leadership changes within their Gemini team now that they've been surpassed by 3-5 open weight models and risk loosing frontier status all together in the near future.
Open weight models aren't likely to be open weight in the long-term. China has started considering export controlling and limiting access to model weights [0].
They don't need to be open weight in the long term; once there's an open-weight Fable-level model with 1M context it'll be pretty much good enough for all coding tasks, no need for new models.
Google somehow managed to snatch defeat from the jaws of success with their AI products.
They literally forced me and my company out of Antigravity by phasing out AI Ultra subscription without any proper product follow-up. Antigravity IDE cannot even have poweruser subscriptions now from Google Workspace an Gemini Enterprise Agent Platform cannot be attached to Antigravity IDE.
Gemini Enterprise Agent Platform has an incredibly abysmal setup process, and if I want to limit spending per-user I have to create projects per user. The fact that you cannot activate Anthropic models on it if the billing still has free credits is almost a joke.
I was a big proponent of Google and Gemini, but they left us reeling with their abrupt product decisions. Forced us to buy $200 subscriptions directly from Anthropic/OpenAI.
Also a big proponent of Google and Gemini, but their stubbornness in artificially splitting their consumer and enterprise products is extremely annoying. It's pretty weird that I have access to more powerful tools when using my personal Google account compared to my corporate Google Workspace account.
It's literally the meme of the MS org chart pointing guns at each other.
The GCP team wants their slice, the other team wants some otjer slice, and so on. Everyone wants some crap for their promotion package.
It's no wonder Meta has shit the bed even worse.
It's also why Google still releases actually decent, useful models despite the product being such a hilarious mess. A lot of the time Gemini models have actually been better as production LLMs as part of LLM-based production applications than OpenAI and Anthropic models when it comes to the complete cost:quality:latency:adherence picture. And they still are. We have products in prod that use Gemini because they're better than any other model at the specific task. But we wouldn't dare use it for anything coding related, or even just as productivity tool to rely on, because as a consumer product it's a joke.
> The GCP team wants their slice, the other team wants some otjer slice, and so on. Everyone wants some crap for their promotion package.
I got a Google One plan for Gemini, but it came bundled with YT Premium lite, and that somehow made it impossible to renew YT Premium for 30 days. I suspect different teams stealing customers from each other.
Google also gave away 1 year Gemini plans with Pixel phones that either did not work at all for existing Google One users or messed up subscriptions by downgrading your account to worse plan, or making your existing paid time shorter if you been on cheaper plan or recently changee countries. Etc.
Like when you try to give Google money they try to squeeze you as much as possible.
At the same time you can get 5 time more limits for free just by registering 10 free Google accounts.
Google subscriptions are one big mess.
As someone who's been using Workspace as a personal email account for over a decade this has been such a struggle forever. Just lots of odd limitations to feature sets all over the place.
When they swapped Google Assistant for Gemini as the default voice provider in Android Auto it was so annoying. My wife's non-work space account can get Gemini to do the normal things like play music and what not, but my Workspace one can't do much of anything at all. I can talk about nearly any random topic with it, but getting it to change the playlist, nah, can't help you there.
It's no surprise to me to see them fumble actually supporting a lot of the consumer features of Gemini into Workspace.
I'm in the same situation. But I was shocked discovering it goes both ways: many new Gemini functionalities are only accessible using a consumer account instead of a Workspace account. Also, Gemini is now the only major AI assistant with no support for MCP connectors. Instead of adding this to the core product, like ChatGPT and Claude did, somebody at Google decided that it was smarter to add this fundamental feature to a new product instead: for enterprises this is Gemini Enterprise (which is a product completely different from the Gemini app); for consumers this the new Gemini Spark agent (meaning that you can use MCP within Spark but not within a "non-agentic" chat)... It's clear to me this a symptom of Google shipping their org chart, which is a disaster from a product perspective.
I currently have a free trial AI Pro subscription that will run out next month.
If it weren't for the $10 GCP credit, I'd straight away cancel it. I don't see enough value in Gemini to justify the $20 subscription.
Maybe because they want to train on your data? Workspace AIs are not trained on your corporate data.
i think Google would see more success if they kept the CEO and everyone at the bottom (ie. doesn't manage anyone), and fired everyone else.
Build a whole new management tree - the current people all do a terrible job.
This drove me bonkers. You can enable play music (etc) in Android Auto for workspace accounts by enabling apps in Gemini. From memory (looking at the settings now, not 100% sure of the magic steps required), but go to admin.google.com, go to 'generative ai', 'gemini app', and 'apps settings', then turn on 'other Google apps'. This lets you play music (and other things) in Android Auto.
Also have a workspace as a personal email and ended up getting a personal gmail just to try out the subscriptions before I gave up.
I have multiple anthropic and OpenAI max plans. For Gemini I just use my Cursor $200 a month plan (which also gives me the ability to try grok, conductor, etc)
The entire Google Workspace division is basically Microsoft. I assume they are making a lot of money in order to support their continued disfunction.
It's absolute insanity. They have all the resources to have been able to lead from the front with this new technology. They have the products and users to integrate this technology into people's already existing lives. But they keep fumbling.
They're not even benchmarking against other models now, just against themselves - which tells you everything you need to know.
What does "leading from the front" get them?
There is absolutely no loyalty when it comes to coding. Nothing could be more common than people threating to jump ship whenever another frontier or open source model comes.
Google is clearly able to keep growing their free and consumer and small business use cases. Unlike corporate coding, we actually have evidence that solo and small businesses can actually see productivity gains.
Anthropic and OpenAI need to stay dancing like mad, because it's their source revenue which underpins their investments.
Why does Google need to shove something at the top at the same desperate cadence? Other than "recursive self improvement leads to AGI" it seems perfectly fine if they push out something dramatically better every year and half.
Unfortunately them giving up coding means they have less traces to train on.
Google promotes people based on shipping features, not making good software. So this breakage means someone is shipping features.
You should be happy for them.
They simply don't have the leadership to lead. And it starts from the top.
I trust Google to:
* Abruptly ban me and all users from a Google Workspace for no reason whatsoever.
* Abruptly shut down a service.
I'll never give Google my direct money.
> Google somehow managed to snatch defeat from the jaws of success
This is still very early days. Who is "on top" has flipped back and forth many times already. The next frontier model release (from whomever) will change things again.
I don't think it's that early tbh, agentic coding has ~90% adoption in the US.
Claude Code has largely won individual developer mindshare and has been on top ever since it came out. The benchmarks change, but almost nobody opts to use anything other than Claude IME when I ask them. Enterprise is more competitive since they care about costs and other things, but developers leaning towards Claude puts a thumb on the scales there.
The product doesn't have much lock in, so it is possible to dislodge Claude, and Anthropic could (and some may argue is likely to) just shoot themselves in the foot again and again and again, but Google has never been particularly good at enterprise sales, and they have never actually been at the frontier of intelligence.
I think Google's incentives have mostly about building models for their products, which makes them focus more on the cheap end, and while they need that, it feels like the Innovator's Dilemma is biting them here.
I own a lot of Google stock from working there in the past and have been quite happy about their trajectory up until the last 6 months, but I am getting pretty antsy about their AI story these days.
> Claude Code has largely won individual developer mindshare and has been on top ever since it came out.
Claude Code's success is not due to the agent but because the model is considered the best for programming and is very heavily subsidized, compared to pay as you go API prices. Consumers and Enterprise are not really locked in and will go where it makes the most sense.
I think they have almost no loyalty by actual developers.
There is no reason to be loyal....There is no moat.
Basically you may choose to drink brand A water bottle, brand B water bottle or tap water. Oh and you might choose the glass water bottle if you use API/Fable.
There’s no reason to be loyal, but I guess it’s a bit like any tool, once you get used to how one works why would you change to another? There is some stickiness with an LLM + harness.
Exactly developers can switch to another coding cli and the learning curve is close to zero. Mindshare without switching costs is just a popsicle in the sun.
Every model has its strengths and weaknesses, being loyal is suboptimal unless you mean being loyal to all of them, which is why cursor would have been well positioned before it got acquired. Now you have to jump through hoops to call Gemini from Claude from codex. Yuck.
Claude code was one of the first agentic code tool and when openai release models similar in performance they didn't do as well in their tools (now codex)
Claude Code (or any other model/infra/harness co-design) is not subsidized in any normal use of that word. It's trough filling (I've written this up in lurid detail so I'm only going to do it again if anyone cares).
It's not true, it's just a play for margin.
> I don't think it's that early tbh, agentic coding has ~90% adoption in the US.
Where does that 90% figure come from?
Early days or not, Google fucking deleted my IDE and wiped my settings. It took them days to roll out a fix, by which point I had migrated off Antigravity.
I guess? Maybe I'm alone here but I don't feel like Fable is particularly more useful than Sonnet most of the time. I feel like the LLMs are good enough for the majority of uses and the hyper expensive premium ones are way into diminishing returns. At this point with Kimi being as good as it is, if they jack up the price any more I'll just go open source.
I don't think that's true if the reason a company has left the vendor for given model by making it hard to buy. Enterprise IT is enough of a pain in the butt that people will forego the new shiny to avoid the old painful unless it's genuinely better. As you say though, the best frontier model flips regularly, so companies won't go through the hassle of deploying a model if it's proved horrible to do in the past. They'll just skip that model because their current one is fine.
Model quality is only one aspect, the bigger problem is making it work in a fully integrated enterprise platform, and Google has always been lacking when it came to the latter.
At this rate, if Google has a flagship model, you're better off plugging it into a competitor's tooling than hope Google figures out how to use it.
Aren’t the subscriptions extremely subsidized and burning cash for Anthropic and OpenAI? A reasonable explanation is they’re simply abstaining from the war of attrition, especially given cheaper comparable models are breaking the illusion that the “frontier of intelligence” has any kind of per token margin.
This is hotly debated and completely unclear. Let's say Anthropics Opus models cost the same to serve as GLM 5.2. GLM 5.2 is 4.4$/MTok while Opus is 5.6 times more expensive. Assume that GLM 5.2 is served at essentially zero margin. Then Anthropic has >80% margin on API pricing. So even if an average person with a subscription pays only 20% of the API price of their usage, Anthropic makes money on subscriptions.
And the real numbers could be better for Anthropic. It's feasible Opus models are actually cheaper to serve than GLM 5.2 because Anthropic have optimized the hell out of inference.
Sure, but then why wouldn’t I use GLM 5.2 at cost or K3?
I guess that’s the big question, will people pay a big margin long term to use their end products / models or will AI tokens be commoditized by many competing players. For coding if I had to pay API costs I’d switch in a heartbeat, enterprise maybe more reluctant?
The subsidizing thing is repeated over and over without proof. Personally, I doubt they're actually subsidized.
Possibly, but aren't the tech giants positioned to win a war of attrition? Then again, they're more likely to sit on that cash and wait for the opportune time to buy a frontier lab.
Makes sense, subsidizing tokens doesn’t seem like a great strategy for a public company.
And Google alway has a target on its back for antitrust (regardless of claim validity)
Google Gemini agy is not allowing you to use your token via your own harness. My own harness is far more efficient than agy. They can take a simple stance - if you exceed your token limit they block you - with the 5 hr limit they are already doing this . so there should be no reason to block you from using your own harness - if you are more efficient - you gain - if you are less you lose .
They are not allowing me to hit their endpoints which agy hits - it's frustrating . i tried to hack it with gemini itself. what i love about gemini is it's so encouraging and ready to help you - even against the agy client : ) .
Even though im so frustrated with this - i still love Gemini for some reason ! Most encouraging model in the world!
> I was a big proponent of Google and Gemini, but they left us reeling with their abrupt product decisions.
Likewise. This seems like a common feel. I have at least spent $4000 and likely a lot more on Gemini API because I really wanted them to win. I gave up.
I am going to ask a very direct question and only because I am curious.
Why do you care? Why would you spend your own money to a multi trillion dollar company so that they win their own "war" against another multi trillion dollar company?
Please don't get me wrong, I know the question can seem a bit negative, I am really just curious.
No, it's a fair question. The answer: I believe(d) in Demis Hassabis's vision of AI
Although, I think saying 'wanted them to win' was not accurate. More like, I stuck with them hoping it will get better, and it did get better in many ways, coding was not one of them.
> they left us reeling with their abrupt product decisions
This is what happens when you put a McKinsey consultant in the role of CEO of an organization where product managers run the asylum instead of engineers.
i've got to wonder how much of this is intentional, and how much of this is just google being their usual terrible selves at anything consumer-product related.
Google's biggest and most important customer for all this AI stuff is google. Do they actually want other customers, or is having other people use their AI just an annoyance at this point, where we use up compute that they'd rather use internally...
+1 same way vscode killed its usage base and gave it to its fork - cursor
They'll probably be back later. It's very possible they are "saving up" for a much bigger run.
More likely an acquisition.
Snatching defeat from the jaws of victory is the specialty of product managers.
It's funny that they triggered the infamous "Code Red" moment in OpenAI when the 3-3.1 models came out. I switched to using them for a lot of single-shot LLM calls because they were fast and cheap. Their only area that they were lacking in was agentic/tool calling.
Needless to say 3.5 was a disappointment. Curious to see 3.6.
> Google somehow managed to snatch defeat from the jaws of success with their AI products.
HN lives in a bubble.
I have German/Italian/Polish clients virtually all use Gemini and NotebookLM. Talking insurance, banking, consulting, legal.
The real world doesn't look at pointless benchmarks on writing react tailwind crap, they are already google suite users, get the tools, test them and adopt them, end of story.
It's going to be like with angular, never mentioned on the net, widely used in the real world.
I have friends and family who use Gemini, but entirely because their Pixel phones came with a year of it for free. No other reason, and they will most likely never pay for it.
They literally forced me and my company out of Antigravity by phasing out AI Ultra subscription without any proper product follow-up
That sounds awful.
For those of us who don't follow the AI hype cycle, what does that have to do with the topic of this thread: Gemini 3.6 Flash, 3.5 Flash-Lite, and 3.5 Flash Cyber?
That's kinda like asking why someone might complain about Chrome's memory usage in a thread about a new version of V8.
"Your new models are great, but uh... How exactly am I supposed to use them?"
That is, given a lot of users' contexts, the way they can and the way they want to use these models are increasingly disjoint.
From the posted link: "3.6 Flash: Our workhorse model that delivers better coding, knowledge work, and multimodal performance."
What is Google's recently released AI coding product?
You don't see what Google phasing out an AI product subscription has to do with a Google AI product release?
I wonder how big the Pro model is that Google is using behind the scenes to train these smaller ones.
Going on baseless speculation, the lack of accompanying pro models with these flash releases either means: 1) the model is too big to be economical, 2) google doesn't have the compute to serve the big model, 3) their big model has too many alignment issues to serve to the public.
edit: looks like benchmarks are up on https://artificialanalysis.ai/models/gemini-3-6-flash. It's solidly middle-of-pack. However, if you want to be most fair to flash, look at the intelligence vs time per task and intelligence vs outputspeed benchmarks. This is a very fast model.
edit 2: I use antigravity from time to time and in my experience, 3.5 flash is an underrated model, so long as you know what it's good for. It's very good at frontend (much better than gpt 5.5) and it's fast, so it's a great tool for iteration. I expect 3.6 to be no different.
I choose fourth option.
4) googles big model just performs worse than K3 and GLM so they choose not to embarass themself.
Like I love Gemini and use it a lot to one-shot whole MR with huge contexts, but its just much worse when its come to tool use and agentic coding.
It's also very possible that they know their big model underperforms chatgpt 5.6 and fable by too much, so they are focusing on what they can get wins in like speed instead.
That and/or the business case isn’t as clear when serving enormous models? You’re constantly stuck in a red queen’s race where your profitability window is increasingly measured in weeks because the Chinese are right behind you.
For small models (which are probably distilled from their big ones) you can serve them economically all the time and not hemorrhage money.
Yes agreed - I wrote this up a while back https://martinalderson.com/posts/whats-going-on-with-gemini/
My view then was they are optimising the models for inference ability on their own hardware AND use cases, which is often speed and time to first token.
They've somehow seemed to end up with terrible compute shortages, which again is surprising given how good Google is at infra deployments AND have their own hardware. From rumors out there they are turning down enterprise deals for Gemini because they don't have the compute.
The problem is they're falling further and further behind on frontier class on coding especially, and since I wrote that article it's got even worse with open weights models undercutting them on price AND intelligence.
There was some recent reporting that a July release of the Pro model got pushed back for exactly that reason. Its performance was not good compared to the OpenAI/Anthropic big models. They are having a lot of problems with posttrain.
This is the feeling i get too. Cant produce quality, but can produce something that is super fast...so take the wins where they are.
We don't have enough fast models, so I see this as a positive. I just test drove Gemini Flash Lite and it's crazy fast.
> focusing on what they can get wins in like speed instead
Speed as a differentiator has always been Google's thing. They (used to?) show the microseconds it took to query & rank web-scale search results. Chrome, notoriously, focused on speed at the expense of resource use. The very many efforts to efficiently speed up Android & its runtime since its inception, and so on...
> their big model underperforms chatgpt 5.6
Possible but TFA claims:
I personally doubt that.
It would be a shame if they cannot beat Kimi K3 or Qwen3.8 Max, both of which are claimed to be Fable-like. If that is true, it will be [or would be] the first time a major American lab falls behind a Chinese competitor.
I wonder if the broad use of AI overviews on Google search results is having an impact. Maybe the numbers make it more profitable to use their compute on several billion searches a day rather than selling API access.
I think it's a safe bet that Google seems more interested in making a model that improves Google rather than making a model that improves workers.
Fast, light weight, ok intelligence. Perfect for serving 20B+ prompts per day mostly surrounding banal human things.
OAI and Anthropic's cloud spend can cover the revenue gap, as Google is already capturing a large chunk of those guy's revenue.
AI overview is just a summarization of the top 2-3 results. Of course at Google scale that will still need a ton of compute, but the requirement for generating an overview is many orders of magnitude lower than asking the same question in Gemini.
Not to mention internal use cases, such as prediction-related tasks like serving ads.
The AI mode on Google search is pretty impressive. Helped me figure out what a bunch of stuff I was seeing out the window was while traveling.
Microsoft also seems to be working in this space. They recently released this:
https://huggingface.co/microsoft/bitnet-embedding-0.6b
It’s a small multilingual embedding model designed for things like search, RAG, and semantic similarity. It supports a fairly large context window and is designed to run efficiently on a CPU in a GPU starved world.
The interesting part is that it builds on BitNet, using ternary weights of -1, 0, and 1 instead of the usual floating-point weights. That should make indexing and searching large amounts of text much cheaper without giving up too much accuracy.
It seems like there are some credible rumors that Google is actually winning in terms of actually building models that work and don't lose money- between how they're able to price them, the TPU advantage and their capex advantage (being able to raise debt + just having a lot of cash - well I said not lose money... more like not go bankrupt).
From the outside they look like they're behind in terms of frontier models, but I think they might be the best positioned to not go out of business when the bubble pops.
Also look at the fact that they've been able to deploy AI-assisted search at google scale. It must be another order of magnitude larger (at least) than the model deployments for OpenAI and Anthropic.
Of course unless you're inside Google it's impossible to know for sure.
That "TPU advantage" might be slowing Google down (though likely not as much as their internal bureaucracy).
Porting CUDA-based research, debugging, and overall experimentation speed is likely slower.
The GPU is still king for training.
In terms of open models, Gemma 4 beats the pants off everything else to the point that paying for APIs becomes hard to justify. Qwen has the meme-share for coding, but it feels much less well rounded. I have no doubt that Google have both the infrastructure and the expertise to curb stomp everyone else, should they resolve in earnest to do so.
Lest we forget, "Attention is All You Need" came from Google.
They basically don't exist in the currently most profitable LLM market (coding).
Yes, subs like codex are heavily subsidized. But API billing has massive margins and that's what enterprises pay.
Does it have "massive" margins? Afaik no one has said publicly what margins there are on an API call?
Logan Kilpatrick said on an interview not too long ago that flash 3 and 3.5 are the same pre-train. all gains on top of 3 flash are post-training
Maybe, but they said they have “started” the Gemini 4 pretrain. So not having done any significant pretrain in a year or so seems odd to me.
Maybe it's like Meta not releasing the big version of Llama 4 a year or two ago
I wonder if they waited for the new TPU generation to train a larger base model.
"3.5 pro is testing with partners! will hopefully land soon."
https://x.com/OfficialLoganK/status/2079596415509303596
> the lack of accompanying pro models with these flash releases either means:
Rumors say 4) it didn't perform well, especially in coding so has been delayed
Or perhaps 4) it's outcompeted severely by other models & releasing it would only tarnish their name
It's scary relying on Google's models.
I have a very price sensitive workload that used to run on flash 2.5 lite - it's deprecated now.
The replacement 3.1 flash lite is a lot more expensive, but now also has a sunset date.
3.5 flash lite is even more expensive.
So the price is rising and you have no choice but to keep paying more and more.
I moved directly from 2.5 flash lite to deepseek v4 flash, its already cheaper and if your prompt caching is good you can save so much more money.
could you explain how to optimize prompt caching or point to a doc about it?
Anything Sam Rose is worth reading: https://ngrok.com/blog/prompt-caching
but the implementation will be up to your provider and harness, for deepseek, they expose some numbers: https://api-docs.deepseek.com/guides/kv_cache/ and Anthropic has a list of actions invalidating your cache: https://platform.claude.com/docs/en/build-with-claude/prompt...
Basically, you avoid anything dynamic: model change, tool change, etc it's also important that your system prompt or main prompt doesn't have non-static data like the date/time/place or someone's name (the person you interact with in a chatbot for example). That should be left to tool call or search.
Sam Rose here. Thank you <3
samwho? samrose.
The man himself, thank you for the articles =)
You are extremely welcome.
I just put the varying parameters in a trailer prompt and have them change every time. It doesn’t matter because the cache is prefix keyed. You lose caching for the last 20 tokens or so but that’s not a big deal. Moving it to a tool call makes it too slow (needs full roundtrip).
If you’re constructing the prompt you don’t have to jam everything together you can arrange it appropriately.
Yes indeed! Mostly don't put changing data in the beginning or prepend.
Not an open source, but I discuss it in my book with examples for OpenAI/Anthropic/Gemini, https://crimede-coder.com/blogposts/2026/LLMsForMortals.
All of the models, you need to have a consistent input to get the cache hit. So if you are chatting with a document, and change the system prompt, it will be a cache miss, even if the rest of the items are all the same. If you even pass in the document in not the same order as the prompts, it will be a cache miss. Or if you add tool calls or structured outputs, it will be a cache miss. (Since those generally go at the beginning of the prompt call, not at the end.)
Most of the time when reading documents from URLs directly it will never cache. (Need to typically pass in the bytes directly, or use the provider document store index.)
Gemini has a 4096 minimum token size with the 3 version models before even getting a cache hit. OpenAI it is lower (1024), and is automatic, but only happens in increments of 124. Anthropic can also get cache hits at 1024 tokens, but you need to explicit ask for it (and pay extra).
Caching by default typically lives for 5 minutes since the last cache hit across providers. But some of them you can ask for longer. AWS for Anthropic models can be tricky with multiple endpoint routing, so can get cache misses if it happens to route to a different endpoint.
That’s part of why, since Firebase, I’ve tried to never depend on Google products for business, especially not GCP.
Features stay in Beta for ages, whatever that actually means, and released ones get deprecated things fast.
Where some of the competitions treats deprecating entire services as "let’s not put it on your frontpage, put deprecation notices all over the doc, and politely ask new users not to start new project with them".
All models are increasing in price. Everything up to now has been subsidized by investors, private and public.
They know that there's big enterprises that will have a strong preference to work with another big enterprise instead of relying on a younger company. At least that's why I think they believe they can do this sort of thing and get away with it.
Just switch the model, its not that much effort tbh. And u can also get a cheaper model than 2.5 lite for the same intelligence
its not always that simple. dropping in a new model is trivial, but highly specific workflows may rely on specific _invisible_ aspects of a model. when that model gets deprecated, the workflow needs to be rebuilt/re-tuned to work with a different model.
google's inability or unwillingness to provide stable timelines for model deprecation makes it risky to build complex workflows using their models
Load-bearing (whoops) quirks were noticeable months back, but haven't most flagship models become predictable and reliable?
100% agreed in the same boat right now. Feeling really screwed over by Google rn
You would be surprised how much of a difference the model makes for certain niche tasks.
For my use case, `gemini-3.1-flash-lite` is ~20% higher accuracy than the next best model of comparable cost (considering both proprietary and open-weight alternatives)
Well it is a bit surprising that 3.1 flash-lite could be better than deepseek-v4-pro (cheaper output and way cheaper cache so might cost less for quite a few use cases).
They are not anywhere close according to pretty much every benchmark (even v4-flash is considerably ahead and its way cheaper than flash-lite). Maybe tuning prompts/tools/etc. might be useful?
"Intelligence" being what, math? Coding? Unfortunately there's a billion use cases for LLMs whose performance is not at all captured by the popular benchmarks they're all trying to maxx.
if you are relying on a model for a business process, it should be simple enough to benchmark on that process
> So the price is rising and you have no choice but to keep paying more and more.
I presume you can't use deepseek?
There are plenty of 3rd party providers hosting deepseek models, if you don't want to use the 1st party API. 3rd party providers are generally slightly more expensive, but still quite cheap compared to other models of similar vintage and size.
sadly it's not multimodal
same here. our production workloads was on Gemini for 2 years. seeing Google unilaterally dropping perfectly fine models and charing you 50x more for worse results is not good.
we are switching to Deepseek.
Opencode Go is just the same. Each month I will I can do less. Dont ask me why?
I felt the same way about openai's text-davinci-002 and code-davinci-002 (gpt-3.5). They were amazing completion models and openai basically dumped them with no equal cost or equal performance replacement. Instead all their models are opaque with no ability to work in completion mode where one actually controls the text input to the model.
These days no company even has completion models where one controls the text input fully. Worthless.
I'm running price-sensitive data extraction workloads on flash 2.5 and its still the king when it comes to accuracy + cost, all the gemini 3 variants perform a bit worse and cost a lot more. Low-key freaking out, ngl
>So the price is rising and you have no choice but to keep paying more and more.
You can also just write code like you did a year or two ago.
same I just switched to OpenAI after using flash 2.5 lite for almost everything at our company. We spent thousands just to build this workflow now Google says screw off
It's a bit disheartening to see no comparison to other models here - and I'm not sure this pushes the curve anywhere. 3.6 flash is more expensive than GLM 5.2 - but seemingly worse, although this post is really light (lite?) on details.
It seemed for a time that Google had finally gotten the ball rolling, but I'm doubting that more and more as time passes. We'll see what happens with 3.5 pro I suppose.
Here, my comparison of 3.6 Flash vs Sol vs Luna vs Terra: https://aibenchy.com/compare/google-gemini-3-6-flash-medium/...
How does your comparison work? It places Gemini 3.6 Flash Medium above GPT 5.6 Sol High and Fable 5 Medium, which makes me skeptical because that... would be making headlines that I'm not seeing right now.
I have created various questions/tests and put the models through the same tests.
I record whether the answers are correct, and the generation stats (costs, latencies, tokens used, etc.).
I have no idea why the Gemini models do so well.
I have recently added new tests, whose sole purpose was to find some cases on which Gemini 3 Flash fails (I don't like cherry-picking models or tests, but I also find it strange Gemini Flash models leading in accuracy). I made a more complex coding/tool-usage test, that I expected it to fail, it did fail it once locally in my debug tests, but when I finalized the test and ran the entire testing suite for all models, somehow Gemini 3 Flash still got it right...
Gemini models are REALLY intelligent (and they are actually my favorite model to use via the chat app to ask questions), but they somehow fail in real-word coding tasks where they have to modify files, check results, debug, etc.
My tests harness provides a lot of mock data, and limits the number of actions a model can choose from. I am starting to think that maybe the models are not bad, just that the coding harness are not optimized for those type of models, and Google doesn't really provide their own "Codex".
Interesting, well it'd be interesting to check out some individual examples where Gemini beat the others.
Also, would be great if you could add GPT 5.6 Sol XHigh and Fable 5 High as well, just to see if at least those beat Gemini which is currently your #1.
I don't like to divulge tests, but one of them is a chess puzzle.
> would be great if you could add GPT 5.6 Sol XHigh and Fable 5 High as well
I would like too, but I avoided them for several reasons:
1) Cost - this is a hobby project, those models would cost tens of dollars for each benchmark run, multiply this by tens or hundreds of models and ...
2) Time - the high models are already taking a really long answer to respond (5-10minutes per question). I run each question with 3 repeats (run the same test three times), so it would take 30 minutes per test. If I change my tests, methodology, or add a new test, it would take a really long time to run the benchmark. Also, I like having results immediately when a new model is released, now I can post within 30 minutes of a model's release the benchmark results.
3) High reasoning usually does WORSE on most tests - if you look at the leaderboard, it's sometimes counter-intuitive, but models with high or max reasoning usually do worse than medium and low. This is because the questions are quite targeted/direct, and the models overthink the question and miss the solution. Or the long thinking context makes them perform poorly. The generation tasks (SVGs/HTML animation) are usually better with longer reasoning, but short code fixes, trivia questions, puzzles, etc. are answered by low/med reasoning with more accuracy in general
Also, Fable is borderline un-testable, it refuses to answer many questions, so it scores poorly anyway.
Gemini scores 21/22 because it answers all tests, and it does them correctly, consistently. The only failed test is I think because it miscounted the lines in a file, when responding on which line the bug was in a code snippet.
Oh, and I've also added weights to different categories, so Coding and Tool usage categories influence the score more. This done both to better account for how most people are being used, and also to reduce Gemini's dominance in general/domain specific knowledge.
So yes, Gemini models are at the top, even if I actually (not proud of it) tried to make tests that actually favour other coding-focused models.
> It's a bit disheartening to see no comparison to other models here
Disheartening, but not surprising: the comparison would not be very flattering for Google.
GLM was twice as verbose running the Artificial Analysis benchmark. So it ends up being more expensive
>verbose
GLM defaults to max effort btw
https://docs.together.ai/docs/glm-5.2-quickstart#reasoning-e...
Not really. Gemini 3.6 Flash actually cost $0.01 more per task, compared to GLM 5.2.
https://artificialanalysis.ai/models/gemini-3-6-flash
All the benchmarks I see put it around the capabilities of Opus 4.8 Medium or Sonnet 5 High.
As far as I can tell it's slightly better than GLM 5.2.
It’s really surprising. When Apple announced the multi-billion dollar deal with Google to power Apple Intelligence I thought great things were coming. Instead we are getting more and more bad news: delayed Pro models and AI leadership leaving. I wonder if Apple know something the rest of us don’t know or if they are already regretting their decision.
Besides Apple apparently making Siri AI model agnostic, the choice to go with Google was almost certainly for practical reasons. Google is a low-risk established player that already has a long work history with Apple. Google also isn't in an existential battle to establish themselves, Gemini still amounts to just another project at Google. There is tangible non-zero risk that either OAI or Anthropic will be gone in 5 years, or will be forced to leave Apple high and dry to save themselves. There is almost no risk Google will be in either such position. And worst case scenario, Google has incredibly deep pockets should Apple pursue a "refund."
What Apple wants out of Google is Siri that runs at 8gb ram and isn’t a horrible embarrassment that feels like a primitive markov chain. Given how good Gemma 4 is, Google can squeeze some serious performance in small models. Whether they can make bleeding edge models is irrelevant to Apple.
"Siri, please solve the Jacobian conjecture, and also set an alarm for 8am tomorrow"
As someone on the Apple beta.. the model is almost completely irrelevant to the experience. Apple has gone and done Apple things by nerfing the experience so completely that almost any model in the past year would be fine. I still reach for ChatGPT/Claude/Grok constantly instead of the AI toy that Apple calls the new Siri.
The one thing I've found google's models to be the best at is proofreading text in non-english languages. Probably because I imagine they have the most training data for it as Google probably has the most complete archive of the internet.
> really light (lite?) on
Light. Lite is product marketing seepage.
Yeah that’s the joke :p
Sorry, it went right over my head!
Pelicans for 3.6 Flash and 3.5 Flash-Lite (Cyber isn't available to me through the API yet.)
https://tools.simonwillison.net/markdown-svg-renderer#url=ht...
I am growing tired of these pelicans posts every time a new model is published. Feels to me like low effort personal brand promotion. Just sharing my 2 cents.
You and a few other people, but enough people still appreciate the bit that I'm going to keep doing it.
They're easy enough to skip - click the little "-" icon and you'll collapse the entire sub-thread.
+1. First thing I look for in a model announcement thread. I actually came across this one an hour ago and was sad there were no pelicans yet.
It's a decent heuristic because the better models generate better pelicans. That's all. Nobody sane is going to make a bet on a model based on a pelican. But it's cool, it's tradition by now, and it's a semblance of a good first impression for new models.
I love and appreciate you doing and sharing them, with stats and details.
thank you.
But how else am I supposed to know when we've reached AGI, until I see an absolutely flawless pelican?
All of the pelicans so far have had really weird flaws / quirks so I am always a little interested to see how well these models perform at this task, since I've seen all the past pelicans and have some anchoring.
Seeing a truly flawless pelican would tell me that the model has true visual reasoning capabilities as well as good taste.
I feel the same way. It was fun at first but has gotten tiresome. Does anyone actually use these models to generate SVGs?
Yeah I don’t get it. It tells me which model can draw an svg of a pelican riding a bicycle. It does a great job at that and the presentation is good.
But why is this an indication of literally anything else?
At this point, it's kind of a hackernews thing. Simon posts them as a single comment in the relevant thread. It's okay for this place to have a little bit of a sense of community, and you can just ignore the comment.
Disagree. They're a nice tradition, but besides that, they're a useful way of eyeballing improvements. I realise labs are likely to be training for Pelicans - but if they're all training for them, the differences in the results are as indicative as they were before labs trained for them.
The 3.6 Flash pelican is just about the best I've seen.
Its a nice benchmark. Like hearing the ice cream truck on a summer day.
It's both.
I agree to rednb that at this point it feels like rather obvious brand building, but also, I agree with you that some value is in it.
It does not feel all that authentic though, and it's good to react allergically to lack of authenticity. Bad for a lot of business models, but good for humanity.
Sorry mate, but you sound jealous in all these replies that the Pelican domain isn't your gig. The below is as labored as nitpicks ever get:
> It does not feel all that authentic though, and it's good to react allergically to lack of authenticity. Bad for a lot of business models, but good for humanity.
I hope SimonW keeps them coming.
My ancestors are smiling at me, Imperials. Can you say the same?
More like living next to an ice cream truck car park
Every parent groans haha
At this point it does not show anything as models are fine tuned on all kinds of benchmarks.
I thought the Gemini 3.5 Flash Lite response was quite telling myself. I personally like the Pelican SVG test, to me it is still a charming snapshot of model performance anecdata. No one would argue it's rigorous but I don't think it was ever intended to be.
I get people burning out on the pelican SVG test alongside the rest of the AI burnout, but I guess for myself I'm just choosing to keep enjoying it while I still can.
I think you're underweighting the Pelican test.
Not only does it give you a super easy-to-grok understanding of the model quality just by looking at the image, but when you compare tokens and costs (both input and output), you really get a good, simple COST x QUALITY evaluation across models.
Simon explains it well: https://simonwillison.net/2026/Jul/16/kimi-k3/#what-can-we-l...
Simon, you should put up a summary table page that you update after every release.
My 2 cents: you don’t have to look at the pelican if you don’t want to.
All the models do this well. It's a test that tell us nothing at this point.
Yeah. It's something I can do myself in a couple seconds if I want, also on more varied SVG scenes. If this is going to be a benchmark people turn to I'd like to see more effort put into it than just a one-sentence prompt.
Hard disagree. I love a little bit of whimsy (which I feel the world is lacking more and more everyday) from Simon everytime a new model is announced.
I'm happy for Simon to post what he wants, when he wants. He's earned it.
Vibe code an extension that autocollapses any post mentioning pelicans and by simonw?
I've been close to writing one that will automatically upvote ALL downvoted posts. I'd call it something like Anti-echochamber.HN
Perhaps freshen it up and extend the test by feeding the model the rendered output so it can iterate once. Assuming a multi-modal model.
I love them. Keep 'em coming.
Do something instead of complain
I find Simon's work informative and entertaining; the last thing he can be accused of is low effort. The Pelicans are just a bit of fun icing on top.
I'll split the difference. When it's a blog post there's usually an interesting observation or two, but if it's totally automated? Maybe just do the ones with a post.
I like seeing the pelicans, it's a tradition.
Yeah and it's surely in the training data by now. Long past time to stop.
You say that, and yet 3.5 Flash-Lite produced an SVG without a pelican.
A new model arrives. The pelican, with uncanny commercial instinct, is never far behind.
Sponsored blogs and paid newsletters are after all, notoriously poor at subsisting on silence :)
Linking directly to the rendered markdown as opposed to a post on my blog is a poor way to promote my blog.
A piece of the frame is missing between pedals and back wheel. The frame of the bike passes through the bird. It also puts a cap on the bird's head, and a fish in it's mouth.
The fish and the cap where always added when I asked an llm to improve it's first attempt.
This continues the trend in LLM progress of better=more stuff
I generated a very stylish Pelican using the webapp. Hard to put a judgement on it relative to yours https://share.gemini.google/XSfmve2mEGDV
Flash-lite did the John Cena Pelican
In other good news "the model has been trained to minimize refusals for beneficial uses.".
Otherwise, this news feels like a tiny incremental improvement on Gemini Flash series to make it more efficient with token usage, subagent and cost. Nothing big.
Regarding their benchmark scores on CyberGym, I wonder why they didn't compare their 3.5 Flash Cyber model with Fable 5. I mean they included Mythos and GPT-Cyber, so why not Fable 5 too?
They also mentioned Gemini 3.5 Pro is in testing and its about to become available very soon. Another thing maybe worth discussing is the announcement of pre-training Gemini 4. Sadly, not much technical details to discuss on. Many comments in here seem to mostly be about how Google is behind the others, but honestly, is it really worth the investment to be #1 in Artifical Analysis every week?
Pricing per million input/output tokens:
2.5 Flash: $0.3 / $2.5
3.0 Flash: $0.5 / $3
3.5 Flash: $1.5 / $9
3.6 Flash: $1.5 / $7.5
---
2.5 Flash-Lite: $0.1 / $0.4
3.1 Flash-Lite: $0.25 / $1.5
3.5 Flash-Lite: $0.3 / $2.5
3.6 Flash would be a great model at 3.0 flash pricing. At this pricing, its thoroughly trounced by about 10 models on cost/performance including Grok 4.5. 3.5 Flash-ite would be a great model at 2.5 flash-lite pricing, as is, its trounced by many models including Deepseek v4 Flash.
As is, they are thoroughly outclassed for most usecases. I will say the one area where i do see Gemini punching above its weight class is in tasks that are effectively "Google this for me" / knowledge stuff. So it does have a role, and I do use it. So while I think Google is still in a strong position overall, they are really stuck as a tier 2 AI player right now with text models. They are tier 1 in bio, images, and video.
Could also be that they are pricing it at levels where they actually make money. Without seeing the behind the scenes compute cost on all of these its hard to really judge.
That being said with any open model we of course do know the total cost (or estimate)
3.5 Flash was always too expensive for a "flash" model. They marketed it as "near frontier" level, but there are several order-of-magnitude cheaper open models that compete with it.
In my tests, 3.6 Flash is NOT more token efficient, so it actually ends up costing more than 3.5 Flash, even with the output price reduction.
EDIT: It less less verbose in final output though, but it reasons more.
I assume the optimization comes when you have long-running tasks with many tool calls, and by reasoning more, it reduces the number of tool calls needed.
Am I off, or does Google have the pricing that varies the most between model generation releases?
It seems that they're trying to push up-market, or at least they were.
Given the extremely competitive releases of GLM 5.2 and DeepSeek V4 (both pro and flash), I don't think there'll be appetite for it.
Pricing often reflects what the vendors (expects) the customer is willing to pay. It seems that Google is still trying to find their niche in the market.
A couple tidbits:
> Beyond today’s releases, Gemini 3.5 Pro is currently testing with partners and we plan to make it broadly available as soon as it’s ready.
> We have started our most ambitious pre-training run yet, for Gemini 4, and are excited by the progress.
While them fixing token bloat on 3.5 Flash is good work. That paragraph was the real highlight.
Hopefully 3.5 Pro is soon, and that Gemini 4 can be here end of year and finally have an updated knowledge cutoff.
I guess they meant to release Gemini 3.5 Pro shortly after 3.5 Flash, but then Mythos/Fable and later GPT-5.6 came out with higher performance than 3.5 Pro, so the managers decided not to release it.
Really, what's up with Gemini still not supporting connectors/MCPs/plugins/whatever-they're-called-this-month on web? It makes it a non-starter for any kind of serious use.
It is both less intelligent and more expensive than GLM-5.2, while being closed weight.
It's roughly equal on price and intelligence as GLM 5.2 while being ~8x faster.
But they make up for it by shipping it late.
It's also got vision and audio. So, the better comparison is any of the other large Chinese open models that are better and cheaper than Gemini Flash.
Is that statement based on token price? More and more it seems that $/token hides as much as it reveals. Token efficiency, tokenizer differences, etc. I'm not saying that you are wrong, I am just saying it is becoming a bit more difficult making statements like this without a bit more research.
It's based on the Artificial Analysis "Intelligence Index vs. Cost per Intelligence Index Task" here:
https://artificialanalysis.ai/#intelligence-comparison-tabs
Differences in token "density" are accounted for by pricing per task
It's also about 15x faster.
It’s multimodal though.
It'd be interesting to know how much the Intelligence as a Service angle serves as a value-add in the minds of Google's executives.
You can get decent open-weight models now. That's not difficult. The difficulty is 1) running them and 2) compliance.
My company runs Claude on GCP's Vertex AI solution. We're in the US healthcare IT space, so the models need to be from somewhere that American healthcare agencies and companies have traditionally been okay with sourcing code from - which means the US, Canada, and maybe Europe. The stuff that handles PHI/PII must be in the US. The expense of hosting is more of a PITA than most customers want to go through this early in the technology's lifecycle, and intelligence gains are simply a matter of degree for most business tasks.
In theory, we could find some open-weight model (likely from China) for our development agentic work and host it anywhere you can host AI models. We don't, though, and I think Google, OpenAI/Microsoft, and Anthropic see that as the core of their business.
No word about updating Jules, which is still stuck on 3.1 Pro. I get that it's probably niche but I've really appreciated basically being able to give directions to Jules on my phone, then reviewing and merging a GitHub PR fifteen minutes later. It's been great for getting some progress in on a few personal projects during my commute when I can't exactly pull out my laptop.
Anyone have any good alternatives?
Both Claude and Codex can code in the cloud, it works quite well!
I tested Jules and while the idea is good in theory, I found the model's intelligence to be very lackluster.
Shame. I'm on the $20/mo Gemini Pro plan because the 5tb of cloud storage and the youtube premium lite were good enough perks, and my coding complexity needs were light enough for me to overlook Claude or Codex. But Antigravity is working better than Jules and it's basically giving me a taste of what I'm missing and it's harder to justify not trying out the competitors.
I do not, however I am curious about Jules support. I didn't know if this was a dead project or not. Seemed really interesting but then I didn't see much development/announcements/discussions around it. Last update from their changelog was as you said 3.1-pro support in March.
I have no affiliations with the team or product, but Superconductor reminded me of Jules when I tried it a couple of months ago.
It might be overkill features-wise, but there's a free tier and it likely won't be left for dead anytime soon.
If you own a Raspberry Pi or similar: Hermes + Tailscale + iSH over tmux.
Claude Code
I have a side business selling custom fingerprint jewelry and I use gemini nano banana to clean up customer submitted fingerprint images. This was a step I used to do by hand at 10 - 15 minutes per image and nano banana is the first model that is able to do the task (it is astonishingly good at it). I can't wait to see what the next nano banana can do, hopefully its released soon.
Are your customers clearly informed that you're sending their immutable fingerprints to an AI service?
Yes this is extremely unresponsible if so. Fingerprints are legally protected biometric data in most juristictions.
The mention of an "ambitious" gemini 4 pre-train signals to me that 3.5 pro is probably a lost cause.
That being said, it seems that Gemini is still the best image analysis model, so hopefully 3.6 flash builds on this even more.
Google seems to have anorexia when it comes to model intelligence. They have an internal hard constraint on price per token it seems, and they are trying to squeeze out intelligence with limited compute.
I wonder if there is something with their TPU cycles that makes them want to postpone training a new model. My guess is that they have been on the same base model for 6 months and they may have waited for the next gen TPUs to train Gemini 4, which greatly limits how much intelligence they can increase and forces them to do cost efficiency increases.
Could it be that they have to serve their models to billions of users?
I'd guess they did model-hardware codesign but the design ended up limiting the scaling capability of the model (i.e. they overoptimized too soon).
Google Cloud is probably Google Deepminds biggest competitor. Big company kinda bullshit.
How so?
Unfortunately says more about how competitive 3.5 pro would be today at the frontier if they forgo it for 3.6 flash.
Google has not changed. Following two facts are like tautologies by now.
1. Their AI efforts are very fundamental research oriented. They are really good at it.
2. Their productization sucks. The end products gets little attention compared to competition. It can be canceled at any time. You should never build anything around Google only APIs, AI or not.
Why exactly are they announcing these completely milquetoast models ?
I'd be low-keying the release if anything, given how lame they are compared to their competition.
What am I missing?
I have just tried to switch to 3.6 instead of 3.5 in antigravity and it seems to constantly spit "critical instruction: STOP CALLING TOOLS NOW. YOU MUST WAIT FOR WAKEUP. ". I think I will switch back to 3.5
3.5-lite is the real showpiece here; agentic models of this size are a huge value-add for 90% of knowledge work agent tasks
Kind of excited about this. 3.5 Flash on Antigravity has surprised me recently on a hobby project. When given opportunity to plan, it can deliver on tasks that would take me a while on my own and generates responses at blazing speeds - compared to what I'm used to at work with Opus 4.8 (granted I don't use Opus 4.8 on my hobby projects so just anecdotal). While with Gemini CLI I would just watch it run in circles and run out of 5h allowance before anything useful is produced (or even approached).
Why would 3.6 flash perform a little worse than 3.5 flash on Artificial Analysis Coding Index...
https://artificialanalysis.ai/models/gemini-3-6-flash?intell...
Because AA Coding "Index" consists only of two benchmarks (Terminal-Bench v2.1, SciCode) and generally fails to be meaningfully representative of agentic coding capabilities.
Whats a better option for AA Coding Index?
DeepSWE and FrontierCode are more realistic if you read up on what they actually measure. But the most realistic is to try it yourself. Benchmarks can only vaguely represent typical usage, and how you judge the result. Giving the same real task you have to a few models will make you understand them better than chasing benchmarks.
Could be a good tradeoff for the flash model though. 3.5 -> 3.6 is a tiny bit cheaper and maybe faster?
artificialanalysis.ai has it going from 165 tps -> 304 tps. openrouter.ai needs more data but it has it going from ~100 tps -> ~150 tps, though at peak 3.5 has reached 156tps.
Wow - Google does not even bother to show benchmarks of these models compared to the frontier and Chinese labs - only against previous versions. I'm not surprised. Having worked there for years it was amazing just how inwardly looking the company is.
Pelican svg and a near-perfect 3D MacBook at max effort for $0.16, about a fifth of Fable's price.
Fable 5 still wins on detail with no visible errors, but it's close. And this isn't a memorized pelican;
https://playcode.io/blog/macbook-svg-benchmark#gemini-3-6-fl...
I don't know if it's a rendering error since it looks like your site renders the SVG instead of hosting a static image of it, but the 3.6 macbook looks like an abstract art piece lol, both ff and chrome desktop
I'm more excited for 3.5 pro. Gemini has fallen behind in some areas, but is still one of the best multimodal models.
Has anybody found any models better at image or audio analysis?
Came here to ask basically this. We use 3.1 Pro internally and it's great.
I have no skin in this game and this comment will be gray in a few minutes BUT a friendly reminder that these types of threads are astroturfed heavily by competitor labs and any info should be taken with a massive grain of salt.
Tons of guardrails, lazy model, super confusing plans, expensive 3.5/3.6 flash and lite and 3.5 pro MiA?
Rough patch for google ai
It is 17% more token-efficient than 3.5 and performs significantly better in coding and tool usage benchmarks.
It is also cheaper than 3.5:
> This enhanced efficiency is also combined with a lower price than 3.5 Flash. At $1.50/1M input tokens and $7.50/1M output tokens, 3.6 Flash reduces the overall cost per agentic task, making agents more cost-effective to build and run.
I struggle to see any value in this when DeepSeek is still a thing.
multimodal + latency
LLM reception is truly extreme, even worse than AAA game releases.
Ever frontier lab lived it at least once : missing the frontier by a few months triggers extremly negative reactions, then you take back the lead for 2 weeks, and the hype cycle repeats.
Always happy to see new Gemini releases as IMO Antigravity Pro 16.67/mo plan (Annual) is still the best plan available and have been pretty happy with Antigravity IDE.
If it wasn't for Gemini/Antigravity I'd have to go with a Max Claude plan, as it stands now I can get by with just a Claude Pro plan to get Opus when I need it, whilst using Antigravity as my day-to-day workhorse.
Unfortunately Gemini Flash became too expensive to use as a general purpose model (i.e. for AI features in Apps), luckily there are plenty of cheaper Chinese models to fill that gap now.
Why do you think it's the best plan available?
(Rate limits * capability of the model) / cost
Don't know why are they even pursuing Gemini. Just download the Kimi, call it Kimini and serve it on your GPU. Maybe then train next architecture based on this!
Gemini 2.5 Flash-Lite has been my go to for cheap document processing at scale (especially with 50% off batch mode), but they are really boiling the frog with pricing increases with each version:
gemini-2.5-flash-lite: $0.10 input / $0.40 output
gemini-3.1-flash-lite: $0.25 input / $1.50 output
gemini-3.5-flash-lite: $0.30 input / $2.50 output (a 6.25x increase over 2.5!)
Now watch them deprecate Gemini 2.5 Flash-Lite in the coming months...
How has your experience been with Gemma 4?
gemma4 is the same price as 2.5-flash-lite, and performs better.
Feels like they released this to ride the wave of press of GPT-5.6, Kimi K3, and Qwen 3.8. Doesn't feel like Google has much substance with this post except a bump in version and tweaked their pricing.
I'm a big fan of the Flash-Lite models. They're exceedingly fast and deliver great outputs for high volume use cases where you need to process requests at scale. Can't wait to try the newer version.
For anyone wanting a faster overview: I ran the Gemini 3.6 Flash and 3.5 series release notes through NotebookLM and generated a short video summary. Link: https://www.youtube.com/watch?v=SUFBhvQ2tY4
Looks like 3.6 Flash is the first model with their newest pretraining run (cutoff date is 2026/03), long after 2.5 series.
IMO Gemini has the best free tier models/app for everyday use. Muse-Spark is perhaps just slightly better, but has none of the connectivity to my GApps (for things like “create a recipe in my Google Docs from this image”).
Plus they are probably running these things on every Google search so saving tokens is a huge win for them.
Free? Did I misread the pricing details?
3.6 Flash scores exactly the same as 3.5 Flash on the Artificial Analysis index. Better on some tasks, worse on others. Mostly within what I'd consider the noise window. Looks pretty much indistinguishable from 3.5 Flash, at least on these benchmarks: https://artificialanalysis.ai/models/gemini-3-6-flash
Flash Lite: 0.3/m and 2.5/m
Deepseek Pro: 0.435/m 0.87/m
That's wildly ambitious pricing by Google. You can maybe get away with spicy pricing at the SOTA edge but at the lower tiers everything is a lot more price sensitive.
You need to compare cost per task buddy boy. Cost per token doesn't tell you much when you don't know how many tokens a model will use to accomplish a task
>boy
Seriously?
I deeply wish Google would focus on models like Gemma. Small, powerful, open-weight models you can run on phones or regular computer hardware.
Gemma 4 was released in April. It's a good series of multimodal models.
gemma 4 thinks joe biden is president
Small open source models shouldn't be used for world knowledge, that's not their purpose.
Why not? Seems like a cop out.
Being able to ask questions to small open models seems.... obviously useful?
Because they don't have a lot of parameters to store general Wikipedia knowledge. They're small. Use big models that have high parameter capacity to store general information. Or build a harness around the small model that searches a knowledge base/internet.
Use the right tool for the job. It's like asking why a screwdriver isn't good at sawing wood, or calling C a terrible language because it's hard to make CRUD apps with it.
i mean eventually then will, losing means open source, vertically integrated hardware means you can opensource and win on cost
A lot of disappointment here in the comments, but models like these aren't meant to compete with the likes of Fable or GPT 5.6.
I use 3.1 Flash Lite regularly to classify listings on eCommerce websites. It's great for this task - fast, cheap and accurate.
In fact, it was the single best model we tried in terms of the speed vs accuracy vs price tradeoffs - including the Chinese models.
Of course, 3.5 Flash was more accurate but the 5x cost increase couldn't be justified.
3.5 Flash Lite sounds like it could be a strict upgrade for our use case, without a significant increase in costs or drop in speed.
It's not GPT-6 but it's not trying to be. It's a completely different tool and great at what it does.
So 3.6 Flash is a somewhat of an admission that Google miscalculated by charging 3-5x for 3.5 Flash what it did for 3.0 Flash (3x input and output costs plus large token inefficiency changes) despite only modest improvements?
3.5 Flash Lite is only a hair cheaper than 3.0 Flash, but I think 3.0 Flash is a massively more capable model?
Proof-of-life release while they figure out how to have a competitive frontier model release. My hunch is they pushed too far in the "omni" model direction, that they made something so ungainly, it wasn't as good for normal tasks.
It does seem like their releases are getting closer together. I get the feeling they realized they were trying to roll out to their entire ecosystem and now they’re focusing more just directly on the AI model itself. I think give it a little time and they’ll start to be one of the competitors too.
I was expecting 3.6 Pro. It's been so long since the last Pro model...
They are working on coming up with a better code name. You know, something like ”Fable” or ”Sol”, gotta have one these days. Personally I think they should go with “Mafia”. How cool would that sound? 3.6 Mafia.
"..and in some benchmarks like DeepSWE by Datacurve, we observe up to 65%, all at a lower cost per output token."
"3.6 Flash delivers higher precision with fewer unwanted code edits and reduced execution loops, as seen in DeepSWE (49% vs. 37%)"
So which one is it? 65% or 49%?
First sentence is about token efficiency.
You're right, should've gotten some LLM to summarize it instead of skimming.
Or you could have read it more closely before posting a comment saying it didn't make sense. You know, the old school way.
If I'm going to read it wrong might as well have an LLM to blame.
I read this as a soft let down to not expect too much from 3.5 Pro.
> We have started our most ambitious pre-training run yet, for Gemini 4, and are excited by the progress.
Are they comparing 3.6 Flash to 5.6 Luna and losing? That's ruff.
Why wouldn't they? Luna isn't a Flash model. OpenAI hasn't released a flash-equivalent model since gpt-oss-120b.
Did you ask me a question and then answered it yourself in the very next sentence?
Anyway, given that both Gemini and OpenAI have 3 sizes of models, one would think Google compares their medium size to OpenAIs.
Glad to see the price is going down but it's still too high for a "fast" model
I have liked using their consumer products but they don't make it easy, that's for sure.
never tried gemini for coding, but this news seems to be compelling, i would definitely give it a try
Plugged 3.5 Flash Lite into an existing agent harness that was previously using 3.1 Flash Lite and this shit just does not work. It's not following instructions and is not producing the correct tool calls.
The benchmarks are not particularly impressive. I suppose they needed to release something since the long pause. But not clear why would I use it now.
"We made 3.6/4 Pro, but it sucks, so this is the distilled model" vibes.
That’s the fear.
the real product is the naming confusion we made along the way. Gemini 3.6 Flash, 3.5 Flash-Lite, 3.5 Flash Cyber — at this point even the model cards need a model to explain them
"The model will be exclusively available to governments and trusted partners via CodeMender soon as part of a limited-access pilot program"
we are stealing plutocracy from the jaws of emancipation.
i don't want to live in a world where abundance is guarded and shared among politicians and cronies, whilst the rest are left to rot.
quite a good model, the speed/price/quality ration is a new golden intersection for me, not sure if its as good as grok 4.5 but quite fast/capable model.
So about the same “intelligence” as Muse Spark 1.1 but 2x faster and about 2x as expensive.
It feels like AI is going to be the end of Google. The post-Schmidt company culture cannot produce consistent, consumer-friendly products that any sane person would want to use consistently.
Pretty underwhelming, as expected honestly. I don't want to know what morale is like at DeepMind right now.
Especially when Google owns 15% of anthropic and serves them compute. Double especially when your boss (Hassibis) is also an early investor in Anthropic. Hell his NW might be more Anthropic than Google.
Yep, agreed. They still are not releasing anything frontier-class (Gemini Pro) at this point. Feels to me that they keep getting scooped by others (e.g. Kimi 3) and then are retrenching.
Models are expensive and low performance. On top of that they make you jump through hoops to even use these models without being throttled even for the weaker models. The only reason we are using them is credits. As soon as credits run out we are switching immediately.
Haven't been excited for a Gemini release since December. Wild to see.
Not good enough for high-end, not cheap enough to be for low-end. Next!
Google is walking backwards, with such a pile of cash in pocket, i feel they are doomed.
no actual cyber model release, useless
If only I could use Pi.
Specific to task these can be huge plus point.
2 red flags
1- no comparison with gemini 3.1 pro
2- no comparison with any other model
The model card has comparisons with both 3.1 Pro and other models:
https://storage.googleapis.com/deepmind-media/Model-Cards/Ge...
dupe? https://news.ycombinator.com/item?id=48993130
tl;dr: 3.6 flash is a bit smarter than 3.5 flash, but also a bit more expensive.
My results [0] put Gemini 3.6 Flash at the top.
3.6 Flash high has same $1.5 input price as 3.5 Flash, but output is cheaper from $9.0 to $7.5.
Google said 3.6 Flash is more token efficient, but in my tests it's actually LESS token efficient[1] than 3.5 Flash, so despite the output price reduction, it still costs more.
[0]: https://aibenchy.com/compare/google-gemini-3-6-flash-medium/...
[1]: https://aibenchy.com/compare/google-gemini-3-6-flash-high/go...
I remember back when Gemini looked like it was the best model that this comment section was full of confident predictions that Google had "won" and that no one would ever catch up with them again. The most embarassing part is that I kinda believed them.
if only DeepSeeek supported vision, would never use Gemini.
I'm going to get downvoted/flagged but I feel like we need a new type of "Show HN/Tell HN" etc for "New AI Model Available".
Front page is tedious these days.
If you refresh the Home page, once a day. The top post will likely be 'New AI Model Available '
> I'm going to get downvoted/flagged but [...]
Why do you care that much? Just say it. You're letting an imaginary score determine if you should express your suggestion for improvement. That's a bit wild.
I think it’s safe to say Google seems a bit out of the top AI competition now. The “cyber” stuff also starts to become laughable with open models providing the full power without the crap Anthropic, Google, OpenAI are trying to frontload on you(I.e you are not allowed to develop/review a login system, pay a special cyber operation team to do it for you). They really deserve to become irrelevant in the future of AI.
The silence is deafening.
Google watches over the last few months a flat out assault on the Pareto curve from American and Chinese companies. Release after release pushing the boundaries of frontier intelligence and price/performance.
And the response from arguably the biggest AI research labs in the world by headcount is Flash 3.6.
What do you do when you are given essentially unlimited resources and still find yourself falling behind?
Start your own openrouter.
I wonder whether this is more the fault of Hassabis or Pichai. They are clearly both less capable than Altman or Amodei.
Other discussion from a few minutes earlier: https://news.ycombinator.com/item?id=48993130
Gemini 3.5 flash is already a pretty good model. But, unfortunately, the primary way you can interact with it for coding is through Antigravity - which is actively developer hostile.
It doesn't matter how good the model is if you're (mostly) forced to use it in Antigravity - which turns any model into crap.
Wake me up when Antigravity doesn't suck.
I use 3.5 flash 10x more than any other model, despite have access to all of them. If I'm going to play a slot machine, I'd rather get the pain over with quickly.
We're almost five years into the whole GenAI thing and we're still relying on these guys to spoonfeed us incremental updates.
It's time for them to start focusing on open-weight models and efficiency. Otherwise there's just a layer of marketing hype and "will it do this?" that has to be cut through for evaluation of each and every release cycle.
Models are getting easier and easier to create. The money, if there's any here, is in the harness the user interfaces with, and the data centers running them.
Google seems to be falling way behind the pack. antigravity cli is pure trash. gpt 3.5 pro is now behind and isn't released yet. GPT 6 and Fable 6 releasing next month. What the hell is going on over there ?
> What the hell is going on over there
Google was late to coding agents and as-per-usual fucked it up with their crazy project management culture.
Usually Google gets away with it due to inertia, however this time they are paying a heavy price because they missed out on the training data that Anthropic and OpenAI have gathered with claude and codex.
Just switched to AI Plus from Pro, seems like I won't be missing much.
> we have taken an intentional approach to deploying 3.5 Flash Cyber. The model will be exclusively available to governments and trusted partners
Screw your government! US and Israeli governments should get the least access, but of course we all know they'll be the (only) ones to get full unfiltered access.
3.5 Pro must really suck.
They are comparing against their own previous models instead of competitors. Not a great sign.
At this point just put the Pareto in the bag bruh
is it just me or is this one-upping each other every few days getting ridiculous secreting a whiff of desperation?
Just you. This is typical market competition in a fast moving field.
Some more discussion:
Gemini 3.6 Flash https://news.ycombinator.com/item?id=48993130
whatever, dude. give gemma5
With all the naysayers on Gemini models I'm curious how many people actually use Gemini regularly?
For me, Gemini models are the most usable. Claude Opus and Mistral always try to turn queries into one-shot enormous commits, which just burns tokens, time and annoys me for something which is still wrong more often than not.
Gemini seems far better at listening to instructions and giving me what I actually want, on top of using far fewer tokens and wasting my time. Fable is the only model that's come close to Gemini Pro for me.
And as this is about Flash, it's exciting, I find Flash can usually get the right answer pretty quickly and without too much nonsense.
I keep saying this and people dont believe me, but I have b2b saas systems with actual agents running around the clock, and the performance/stability of the flash model is higher than most other models.
Meaning, its predictable with tool calls, wont spin off a million tools/do weird behavior, its reasonable. Even sonnet in a real world decision making scenario is not reliable, or will reason so long its incredibly expensive.
The benchmarks arent catching all the value, and most people have never actually ran an ai agent in a real context that matters
Who's most people? What are you talking about? Most people here use agents every day and I wouldn't trust flash or pro to touch any important project of mine because they're both terrible compared to the competition, waste of time every time I give them a chance
I mean like an ai agent doing some sort of HR work, not a coding agent. Very few businesses are trusting an autonomous agent.
not a google fanboy by any stretch... though i've been thrilled with the flash line of models... i exclusively use it on high, and have found it to be a great fit for increasing productivity 10-fold while maintaining quality... sure it can't just go off and one-shot a bunch of work, but at the complexity level i tend to work at, neither can the frontier in a robust way that i can be confident in... sure i have to be in the loop more, but that helps keep me grounded and course-correct earlier before wasting tokens... and when you sufficiently spec out a coding/software problem, and i mean really document all of the critical nuance, it will successfully satisfy the constraints... the quality is rarely acceptable on first-pass, but it forces me to stay connected to the architecture more than i would be if using a frontier model... i've found this to be a happy middle-ground of productivity and awareness...
jew
tested the models on aistudio. despite that the knowledge cut off is march 2026 it still knows nothing about 2025!
you can check by asking "list notable world events in 2025, only list unplanned" on aistudio. or you can ask for Charlie Kirk, it also does not know. I tried it multiple time to ensure that I didn't not get routed to older models!
> but google has search
irrelevant, without deeper knowledge about cutting edge technologies or latest libraries, all of it suggestions are crap. even you ask it to search it will still use outdated keyword thus only getting outdated information.
in other word, what a disaster!
At this point, I think google should consider becoming a hyper scaler for anthropic and open ai, and I predict that that is exactly what they do. The model is no longer the most valuable part of the stack.
Google desperately needs to make some leadership changes within their Gemini team now that they've been surpassed by 3-5 open weight models and risk loosing frontier status all together in the near future.
Open weight models aren't likely to be open weight in the long-term. China has started considering export controlling and limiting access to model weights [0].
[0] - https://www.ft.com/content/6049a031-9e9b-464c-97bb-414da04d5...
That contradicts this:
https://www.wsj.com/tech/ai/chinas-xi-touts-open-source-ai-a...
So who even knows.
They don't need to be open weight in the long term; once there's an open-weight Fable-level model with 1M context it'll be pretty much good enough for all coding tasks, no need for new models.