Remember those gpus into chips https://inference-docs.cerebras.ai/models/qwen-3.8-27b , where you have this chip that implements a model that run at those extremely high speeds... the ones that people ask "why do we need that"... if this is true... one of those chips with the correct harness is all that's needed to get fable like intelligence in every single computer in this planet...
Happy hallowen Tax Ramltman and Amowhateverwayyournameiswritten...
That's pretty interesting, because with the Bonsai 2 quantisations it is quite achievable to get Qwen 27B running at 50 t/s on reasonably priced gaming machine [0]. Which in turn suggests that Fable 5 coding performance can be achieved locally with a bit of clever harnessing.
Although the same harnessing should also improve Fable's performance. The important thing here is the cap for local performance looks like it could be quite high.
Bonsai 2 has a bit different architecture as far I understand and speculative decoding is not so great yet. Before that point, original model is faster if you have the memory. Would be nice to see the speed with proper speculative decoder.
Bonsai 2-bit quantization is notably dumber than Qwen 27B at eight bits or even four bits. I doubt you'll get this level of quality even with five of them.
Dumber yes, but not by all that much. If the multiple agent thing really improves the quality by as much as is described here, the quality loss should be nearly imperceptible.
Here the intent is to replicate GVS5H with memdoor a headless coding agent I created months ago. Anyone can reshape the harness by redefining the steps in yaml files , models and temperatures. Up to you guys to try you have the recipe in this repo.
I'd like to see this done with qwen 3.8-flash-next (Q8 or full precision) which beats 3.8-27B on just about every coding/command line use benchmark. In particular comparisons like terminalbench 4.0.
will definitively try it. Overall with a smarter model the improvement is in the token usage and latency. Tbh I think i could get a 10 score if i tuned the prompt and temperature a little more. The original author harness is already giving 9.6 score which is pretty good. However this harness loop can take more than 1 hour on the hardest problem at 100 token /sec. A smarter model such as 3.8-flash-next could get 10 score and resolve the hardest problem faster. That's my estimate i'll give it a try. Thx
Remember those gpus into chips https://inference-docs.cerebras.ai/models/qwen-3.8-27b , where you have this chip that implements a model that run at those extremely high speeds... the ones that people ask "why do we need that"... if this is true... one of those chips with the correct harness is all that's needed to get fable like intelligence in every single computer in this planet...
Happy hallowen Tax Ramltman and Amowhateverwayyournameiswritten...
That's pretty interesting, because with the Bonsai 2 quantisations it is quite achievable to get Qwen 27B running at 50 t/s on reasonably priced gaming machine [0]. Which in turn suggests that Fable 5 coding performance can be achieved locally with a bit of clever harnessing.
Although the same harnessing should also improve Fable's performance. The important thing here is the cap for local performance looks like it could be quite high.
[0] https://store.steampowered.com/hwsurvey/Steam-Hardware-Softw... suggests 16GB VRAM, 32 GB system.
Bonsai 2 has a bit different architecture as far I understand and speculative decoding is not so great yet. Before that point, original model is faster if you have the memory. Would be nice to see the speed with proper speculative decoder.
Bonsai 2-bit quantization is notably dumber than Qwen 27B at eight bits or even four bits. I doubt you'll get this level of quality even with five of them.
Dumber yes, but not by all that much. If the multiple agent thing really improves the quality by as much as is described here, the quality loss should be nearly imperceptible.
I think that requires testing, rather than assuming. There's only so far a harness can go.
GVS5H, which this is based on, was discussed here a week ago: https://news.ycombinator.com/item?id=50228167
Here the intent is to replicate GVS5H with memdoor a headless coding agent I created months ago. Anyone can reshape the harness by redefining the steps in yaml files , models and temperatures. Up to you guys to try you have the recipe in this repo.
This also appeared on HN last week: https://news.ycombinator.com/item?id=50228167, it's the other method they mention, GVS5H
I'd like to see this done with qwen 3.8-flash-next (Q8 or full precision) which beats 3.8-27B on just about every coding/command line use benchmark. In particular comparisons like terminalbench 4.0.
https://artificialanalysis.ai/models/comparisons/qwen3-8-fla...
will definitively try it. Overall with a smarter model the improvement is in the token usage and latency. Tbh I think i could get a 10 score if i tuned the prompt and temperature a little more. The original author harness is already giving 9.6 score which is pretty good. However this harness loop can take more than 1 hour on the hardest problem at 100 token /sec. A smarter model such as 3.8-flash-next could get 10 score and resolve the hardest problem faster. That's my estimate i'll give it a try. Thx