If the benchmarks don't lie, this is getting very close to Opus 4.6 capability - which was the turning point for me for when AI was "good enough" that it became very hard to justify not using it.
I'm sure there's some benchmaxxing going on, and some things you get only with a a larger model.
But I'm feeling pretty confident if not by Gemma 5 than by mid 2028 we'll have local models that are almost always as good as Opus 4.6 was and in many cases far better.
Similar to the way they asked Sol to solve Erdos problems, that's what I want my model to do for programming.
I don't want to try to take my best educated guess at what the best design is BEFORE implementation - especially if you're designing a feature for a codebase you're not an expert in, you don't know like the back of your hand (i.e. one that is mostly or entirely LLM generated).
What sounds good on paper - often times becomes unideal in practice when you get to the reality of implementation.
It may not be worth re-architecting your entire system to get to a "pure" design that would be the best - all things considered.
Instead, I'd like the model to independently design many plausible and coherent good solutions, then implement each of them, then intelligently pick the few winners (after its fixed any bugs that could be causing promising solutions to look artificially bad) - unless there's an obvious one - and then give me the data I need to make an informed decision on which one to go with, all before I even look at the design or implementation.
You're not getting this from a one shot prompt from a 30B model today. You can't even really get it from Sol or Fable - IME. But you can get somewhat close.
This is a harness problem not a model problem, try prime agent it can do that and it will do it well even :P but you need to prompt it in according to its tools and processes.
IME using 5.6 Luna and DS V4 Flash, I notice that although they are excellent at programming, even Opus-like in the way they try to debug, the thing they are worst at is inferring user intent and making good decisions with little information. They are absolutely terrible at that, will misinterpret small wording ambiguities. I suspect that's an ability you can't add with RL training, that it requires the depth of understanding from vast pre-training.
The $1500 Intel B70 with 32GB of VRAM can run this model at max context with good performance, btw. If you don't want to drop $5-10k for running DeepSeek this is your best budget option for local autocomplete/refactor/small scale dev help
The file "Just loads" on llama.cpp, the Unsloth https://huggingface.co/unsloth/Qwen3.8-27B-GGUF is an MTP file, I see mostly the same speed on pp and generation.
There has to be something wrong with those benchmarks, I find extremely hard to believe a 27B model can work similar or exceed opus 4.6.
Worth distinguishing knowledge/task benchmarks from IF / agentic. It doesn't seem out of the question that you can have a small model thats generally good at instruction following and long-horizon agentic, as usually in those cases any requisite knowledge is in the context.
Most of the benchmark improvements afaict are in agentic and instruction following benchmarks.
I wish this can run directly on my RTX 4090, seems like 30B is the sweet spot for dense model to run locally, sadly RTX 5090 is very expensive and I need a new PC and new power supply(and UPS) to run that, adding a second RTX 4090 is another option, but not sure if my PC can do that yet.
even a 3090 will give you the VRAM headroom. i run Q8 on an 3090/A6500 combo. well, Q8 of 3.6-27B. I'm building the Q8 GGUF for 3.8 now, assuming mine will finish before someone else's.
Since Qwen 3.6 27b outperforms Gemma 4 26b in most benchmarks I'm not sure the value - also Gemma 26b is a MOE model whereas this is a dense model, so not typically direct competitors at their sizes - Gemma 4 31b comparison would be interesting though.
If the benchmarks don't lie, this is getting very close to Opus 4.6 capability - which was the turning point for me for when AI was "good enough" that it became very hard to justify not using it.
I'm sure there's some benchmaxxing going on, and some things you get only with a a larger model.
But I'm feeling pretty confident if not by Gemma 5 than by mid 2028 we'll have local models that are almost always as good as Opus 4.6 was and in many cases far better.
What kind of things you only get with a larger model?
Similar to the way they asked Sol to solve Erdos problems, that's what I want my model to do for programming.
I don't want to try to take my best educated guess at what the best design is BEFORE implementation - especially if you're designing a feature for a codebase you're not an expert in, you don't know like the back of your hand (i.e. one that is mostly or entirely LLM generated).
What sounds good on paper - often times becomes unideal in practice when you get to the reality of implementation.
It may not be worth re-architecting your entire system to get to a "pure" design that would be the best - all things considered.
Instead, I'd like the model to independently design many plausible and coherent good solutions, then implement each of them, then intelligently pick the few winners (after its fixed any bugs that could be causing promising solutions to look artificially bad) - unless there's an obvious one - and then give me the data I need to make an informed decision on which one to go with, all before I even look at the design or implementation.
You're not getting this from a one shot prompt from a 30B model today. You can't even really get it from Sol or Fable - IME. But you can get somewhat close.
This is a harness problem not a model problem, try prime agent it can do that and it will do it well even :P but you need to prompt it in according to its tools and processes.
IME using 5.6 Luna and DS V4 Flash, I notice that although they are excellent at programming, even Opus-like in the way they try to debug, the thing they are worst at is inferring user intent and making good decisions with little information. They are absolutely terrible at that, will misinterpret small wording ambiguities. I suspect that's an ability you can't add with RL training, that it requires the depth of understanding from vast pre-training.
The $1500 Intel B70 with 32GB of VRAM can run this model at max context with good performance, btw. If you don't want to drop $5-10k for running DeepSeek this is your best budget option for local autocomplete/refactor/small scale dev help
Oh, can it work with the /v1/completions/ auto-complete endpoint?
The file "Just loads" on llama.cpp, the Unsloth https://huggingface.co/unsloth/Qwen3.8-27B-GGUF is an MTP file, I see mostly the same speed on pp and generation. There has to be something wrong with those benchmarks, I find extremely hard to believe a 27B model can work similar or exceed opus 4.6.
Worth distinguishing knowledge/task benchmarks from IF / agentic. It doesn't seem out of the question that you can have a small model thats generally good at instruction following and long-horizon agentic, as usually in those cases any requisite knowledge is in the context.
Most of the benchmark improvements afaict are in agentic and instruction following benchmarks.
This is one of the most important model releases since most use cases don't need SOTA/Frontier
If you want Qwen3.8-27B Serving Configs for the DGX Spark vLLM NVFP4 and RTX 4090 llama.cpp GGUF I added the setups here https://x.com/ErdalToprak/status/2088299678085308761?s=20
Qwen 3.6 is ~$2/m tok, 3.8 should be drop in replacement. Gemma 31B is $0.34/m tok. The price differential on these models is massive on openrouter.
Where do you see that? From what I can see on Open Router, Qwen 3.6 27B (the closest dense equivalent to Gemma 31) is $0.28/m. Am I missing something?
https://openrouter.ai/qwen/qwen3.6-27b
I wish this can run directly on my RTX 4090, seems like 30B is the sweet spot for dense model to run locally, sadly RTX 5090 is very expensive and I need a new PC and new power supply(and UPS) to run that, adding a second RTX 4090 is another option, but not sure if my PC can do that yet.
even a 3090 will give you the VRAM headroom. i run Q8 on an 3090/A6500 combo. well, Q8 of 3.6-27B. I'm building the Q8 GGUF for 3.8 now, assuming mine will finish before someone else's.
Really excited to see what people do with this. 3.7 27B was probably the best compromise between size and intelligence to run on consumer hardware
I am so happy right now, qwen3.6-27b was an absolute game changer. To see another one in the same league.. phew
Kinda was expecting to see Gemma 4 26B in benchmark comparisons :(
Since Qwen 3.6 27b outperforms Gemma 4 26b in most benchmarks I'm not sure the value - also Gemma 26b is a MOE model whereas this is a dense model, so not typically direct competitors at their sizes - Gemma 4 31b comparison would be interesting though.
I see! Thanks.
This is the homelab model hands down