GLM 5.3 is probably the sweet spot open weights model if you want to go beyond deepseek flash or the new glm flash. I used it with pi and had a fairly good time, especially since it’s less touchy about cyber and whatnot than the US guys. It’s slightly behind Kimi in ability but it’s a lot easier to run it, I’d expect prices (and speed!) from third parties to be noticeably better.
Assuming you’re willing to drop a fat stack of cash on the upcoming Mac m5 ultra with 512 gb unified memory, you can even run it locally, quantized to 4 bit. Whether it’s even slightly reasonable, well, my wife would probably skin me alive but maybe yours is more understanding.
* Models are evolving quickly with high worldwide competition
* Hardware is evolving despite RAM shortages
Is investing a huge sum of money in equipment for local inference a wise use of money? Or are M5 Ultra and equivalently priced local inference hardware future-proof enough to be worth it relative to how the market is evolving? Maybe it’s all a question of what you’d spend otherwise on serverless or dedicated GPU spend…
It is absolutely not worth buying hardware to run models for purely (long term) cost reasons. For open weights models the economies of scale means the cloud beats local significantly and your payback time is like 10 years.
However there are other reasons (e.g. privacy) that might make it worth running locally for some people.
So far I don’t regret buying an M1 Max device with 32Gb of RAM. The models available for it keep getting better (running just about okay for interactive use) and 400 GB/s of bandwidth is still considered a lot.
The models are currently improving much faster than the hardware and this doesn’t seem to have plateaued yet.
That's basically the question I'm trying to answer.
If you're paying Anthropic or OpenAI to use their models, harness, governance, etc., I could see the local inference potentially coming out ahead. They're already starting to ratchet down what your money gets you on their platforms, and that can be expected to continue as the leaders of those companies continue to seek the road to the El Dorado that is being a trillionaire.*
If you're looking to get into the guts of AI development instead of having it handed to you by a provider, that's where it gets murky. I'm wanting to write some sort of agent that does things and get into making outputs consistent in the like, and I'm not sure whether to host something on GCP or buy an M5 Mac.
*Note: El Dorado is a mythical city and many people died trying to find it.
Part of it is knowing that whatever sort of enshittification the cloud providers do, my local programming environment won’t ever be less effective than it is today locally. It’s the same reason my entire development stack from editor to compiler is open source. I don’t need to modify it today, but I always must retain the option to do so later.
There are several things I do in my life that only pay off in the event of a big disaster, like an extended internet outage, civil unrest, supply chain disruption, war, etc.
I like to be able to do the things I do even if offline for weeks.
I spent a lot of money for more flash in my iPad Pro so I can keep all of offline wikipedia and OSM in it, for example, along with tons of books. It’s sort of like being a digital prepper. (Being a prepper is a spectrum, from anyone who keeps food in their pantry to people building bunkers under their house - how much you invest is a personal prudence and threat modeling decision.)
Also, privacy. And when I got the Mac Studio the 512GB was only $15k, which is dirt cheap for that much VRAM.
I have just built an Epyc with 512gb DDR4 3200 RAM for a "reasonable" price and I'm hoping to have a setup with GLM as the architect and Qwen 27b/Next Flash as the implementer. This is 1/5 of the price of the Mac, but also probably 1/5 of the speed lol.
I’ll be very curious what you get with DDR4. I also almost went that way. I have an Epyc DDR 5 rig and the best I see is 10 tok/s. Caveat being that’s at Q8 and a 4090 doing pre fill so it could be pushed up.
The surprising thing for me is how much work you will need to cool the banks if you’re near your memory ceiling. My memory starts soft throttling at about 74C (dies may be hotter, that’s the bank temp) and will turn down speed to try to stay below 80.
Happy to send my llama.cpp config settings if you want it.
Honestly I suspect neither of them will be performing terribly well but with DDR4 3200 RAM I wonder if you'll be counting tokens per second or seconds per token. I mean, you do at least get a lot of memory channels at least, compared to consumer PCs. I am curious to hear what performance you get, I feel there is not enough information out there on what different setups manage to eek out.
The fastest I was able to get my Threadripper 3960X + 2x 3090s + 256GB DDR4-3200 to run a 2-bit quant of GLM-5.2 was 8 TPS. I would expect to be in seconds-per-token territory for a pure-CPU 4-bit quant.
One thing I'd like to try is MoE offloading: I have 2x32 GiB of VRAM and 128 GiB of DDR5 running at 4800 MT/s (only 2 channels though). I've seen people post difficult to believe MoE offloading results albeit a decently long time ago with older models. Maybe there is a quant that would fit with MoE offloading?
That said, I am guessing my problem is not enough RAM - but this poor consumer platform struggles to do memory training with 128 GiB as it is.
Now I surely regret not having gotten Threadripper and 256 GiB of RAM in the before-times.
My measurement was with MoE offloading, but there's only so much you can keep on-GPU with a 200GB quant and 48GB of VRAM. It's hard to overcome the CPU/RAM bottleneck.
For what it's worth, all of my hardware was used; I think, all-in, I'm probably at around 3k-4k USD? Not cheap, but also not the worst for something relatively versatile.
With a 4-bit quant of GLM-5.2, I can get about 0.8-1.1 tok/s on an underclocked dual Xeon E5-2698 v4 with 512GiB of DDR4-2400. I think it was specifically a Q4_K_M quant. Of course, the time-to-first-token is absolutely atrocious.
Which is completely insane for a ten year old configuration.
One could also run it locally on a used dual xeon (or amd-equivalent) server with 512GB RAM, albeit slower, if you have a useful workflow for it that's like "take this day's efforts and run it through various analysis agents", combined with giving it one-shot tasks/modules to build overnight. You would want a place like a garage or basement to put the server because it'll be loud.
Memory throughput performance is better than many equivalently priced (talking about the cheaper end of the used/refurb market here) single socket systems because each CPU is talking to its own banks of RAM.
Using an example of llama-server on a 256GB (each CPU is attached to 128GB) system running qwen 3.8-flash-next Q8, which takes up about 188GB with the model and context, it's not that the inference engine 'knows' about NUMA, but if you use --load-mode mlock to load the whole thing into memory at once at the time llama-server launches, the kernel will distribute it relatively evenly across the memory connected to both CPUs.
Though I appreciate how good it is at "solid" grunt work and at that price (in fact I am paying the grandfathered subscription price; mostly).
I am planning to let go for my Claude AI subscription which I now use only for "planning" and maybe use that via Open Router as PAYG (at to try how it ends up). But god glm is bad at "talking" and "responding" anything prose. Not only quality but it's almost impossible to tune it and make it let go of its habits and biases and enthusiasms which often result in too many too and fro.
So I sometimes wonder at what point that starts becoming the cost and mental hassle. Maybe it's not there for me yet.
Well if you did get the m5 ultra could you obliterate the guardrails and then your wife can ask it pertinent but unsafe questions about how to punish you. Seems doable.
Oh please, as though significant numbers of tech workers on this website are not buying LV, Gucci, and yes Hermes for their pampered wives.
My wife can have an opinion on my tech purchases when I get an opinion on her 20,000$ bag addiction.
And the worst part is both of you should unironically accept these purchases, as somehow luxury bags, high end hardware, AND rolex's often out-pace inflation and are objectively good stores of value in a world of rapidly increasing income/wealth inequality.
and even if you were making such a salary, the quesiton of if the investment on hardware to run llm's locally is still a big if, its OK if you buy the HW cause you'll use it and you get the extra capability as a nice extra, but doesnt make sense to spend so much when you could just get 200$ subs with almost infinite SOTA tokens a month etc (if you dont need the local/privacy aspects of it)
I'd like to ask Sam Altman if he still thinks that it's too dangerous to publish GPT-3. I mean, no one would use it, but what is his reasoning for not publishing it now, in 2026?
There are risks associated with releasing historical proprietary models that were not designed for open release:
- It is trivial to extract samples of the training data that was used, which can bolster existing lawsuits/foster new ones.
- Older models are not as safety-hardened, so it is easier to coax unsafe behaviour out of them, which is a PR risk.
- It may be possible to divulge proprietary secrets from the model (e.g. architectural details that may still be relevant).
For these reasons, and more, it's unlikely that GPT-3/similar models will be released until these concerns are no longer relevant (e.g. when they become a purely historic concern, similar to the open-sourcing of other proprietary software from decades ago).
I think it would be an important historical document as well. We are potentially looking at the dawn of AGI and one of the most important models ever created. Each model is also a kind of ultimate time capsule, containing a snapshot of the entire human collective mind. If you wanted to ask a 2002 person what they thought about future historical events you can just ask them directly.
> If you wanted to ask a 2002 person what they thought about future historical events you can just ask them directly.
The weights arent the truth tho, maybe a timecapsule-vhs but i wouldnt trust llm weights more than more hardcore deterministic media that might get preserved to infer facts from an era.
The companies doing the training are becoming the "winners" that are "rewriting history" as they train their models.
There’s not such a straightforward relationship between safety and model sis.
According to the book The Thinking Game, lower quality models at that time were considered less safe, because they could be easily tricked into doing harmful stuff. In the book, Dario (of Anthropic) was the head of safety at openAI and was responsible for pushing for 10x scaling in training to make the models safer.
It does make sense, a smart model is going to be way harder to jailbreak into telling me how to synthesize anthrax (or something).
Maybe I'm reading too much between the lines, but I suspect the reason is to rub his nose in the duplicity or naivety depending on how generous you're feeling. Publishing the model would be a confession that he was wrong.
AI policy is being shaped somewhat by the things Sam and Dario say. So even if you're not feeling vindictive, it's probably good to keep a track record of the previous things they have said as a Bayesian prior. People who don't know better listen to these people, and maybe they shouldn't.
If you want to dunk on sam for "it's too dangerous to publish GPT-3", you hardly need the release of gpt-3 to prove your point. All the other open model releases already provide pretty good evidence. Moreover the fact that the model wasn't release hardly points to the fact that he wanted to save face or whatever. Most AI labs don't release their old proprietary models, so the fact that gpt-3 wasn't released tells us very little.
I previously posted that DS4Flash was _good_ but not _great_ on two DGX Sparks, but I have to say that GLM-5.3 is pretty amazing. It's been able to tackle all the random hard problems I've thrown at it and it has the intuition that DS4Flash seems to lack.
We're nowhere near a Fable-class model IMO, but things are going to get interesting in this next year.
I'm starting to think Opus 4.8 is significantly smaller than most people assume.
If it's significantly larger than GLM 5.3 (I've heard some insane guesstimates out there like upwards of 5T params or more), that would prove rather embarrassing for Anthropic.
I hear the argument here, but isn't it possible it has dramatically more knowledge and when you get outside the common cases many of us use it for, it'll have completely different capabilities?
I feel like most benchmarks cluster on a reasonably limited area of human knowledge
It seems like there is tradeoff between model size and the need for tool use, which - in my mind - is quite costly in terms of time and tokens.
More detailed world knowledge requires an exponential increase in model size, but most knowledge can be acquired ad hoc using search or database queries. This will fail for questions where the model lacks the knowledge to ask the right questions, but maybe this could be solved by a handful small inquiry models with different knowledge encoded in their weights?
You can't compare models released 6+ months apart. GLM 5.2 was same architecture as 5.3 and not nearly as good. Takes time to build frontier intelligence and distill down to smaller sizes.
Do you use it to write HTML/CSS? Javascript? C++? There's a huge difference in ways people use models and if you are not specific about it then your comment means nothing, unfortunately.
I really like how it doesn't have that Claude talk. It just does the thing without Claude's "load-bearing honesty." It's probably my favorite model to interact with, even if it isn't the best or most reliable.
My second favorite model by now is GLM 5.3 flash which is very capable of day to day task. I use it as the main model and GLM 5.3 for task that is more complex
z.ai coder plan, both in opencode and direct API access. I use it for my side projects like https://github.com/scosman/Biscotti (on-device meeting transcription and summaries).
I have been only using GLM models since last December and have had the best experience without any drama about tokens and geopolitical restrictions. The quality has been great and I am doing more and more with the latest 5.3 and am really excited that consumer hardware will develop in the next few years where I can run these at home.
I have seen you advertise your website a few times. I like the idea of not having to trust the router, so I took some time out of my day to critique your website: https://files.catbox.moe/v68cf7.png
My visit to your website went like this:
1. Visit models page
2. Try to find GLM-5.3-Flash (which is among the ~5 models that 90% of people currently care about)
3. Give up scrolling (which would have taken OVER 50 SCROLLS!!!) and use Ctrl + F
4. Try to find input/output/cached price
5. Scroll all the way up to find out which column is what
6. Notice that output price is cut off
7. Notice that the scroll bar is over 100 scrolls further down the page
8. Use Shift + Wheel to scroll horizontally (most visitors probably won't know this trick)
9. Notice that cached price is missing
10. Conclude that this is probably not a serious offering and bounce
There are probably more issues later on, but this is how far I got.
I would suggest you to:
- Deslopify all pages that a user may visit before conversion
- List important models first (see OpenRouter rankings)
- Move the most important information (model name/input/output/cached price) to the left
- Disaggregate the prices per provider (maybe subtables per model? not sure)
- Measure cache hit rate and compute effective price per provider (see OpenRouter)
(- Optional: Fix the broken link on your HN profile page. Currently, the only way to get from this comment to your website is a search engine.)
Have you guys been having a good experience with OpenRouter? I tried it out recently with Claude, and it cached no tokens, charging me $200 for one conversation of 11 messages.
I tried using deepseek v4 flash with OpenRouter. It switches between providers too eagerly which resets the cache. Then, each provider begins to rate limit me for providing so many uncached tokens, so it just keeps on switching providers. I'm paying for every token... why rate limit me? It was unusable compared to just using the official Deepseek provider which has a much better cache rate.
how feasible is it build a SOTA specialized model for some use case e.g. deal sourcing by using this as pre-trained model or a LORA or similar pattern on top? Gonna shoot my shot at a billion dollar business
Looking at HF, it looks like the unquantized version is half the size of glm-5.2 756 GB vs 1.51 TB. I wonder how they were able to optimize it this well
Is it possible to fine tune this model and unlock / extend its cybersecurity capabilities? I'm scared that maybe we are not ready for an open-weight model with high cybersecurity skills.
It's actually slightly more expensive ($0.50 vs $0.48), but there's a temporary 50% discount.
I've seen dozens of conversations about it in last 24 hours, and every major inference provided added in first 24 hours. I think it's gaining plenty of traction.
OpenCode Go is probably using quantized down DS4Flash. They outsourced to 3th party providers to keep the cost down, and being able to provide that $30 value (instead of the initial $60 > $15).
We saw the same issue with GLM 5.2 when they still published publicly who the providers are on their website. Most ran FP8 but one was doing FP4, so you had this issue where one moment you had the better FP8 and another session you had the FP4 provider.
You can check the internet archive, it was in the FAQ part before they hide/removed it. So if you looked up the providers, and the published quants, yea, ...
Given that a lot of complaints are coming from people that felt OpenCode Go Flash feel like a step down compared to old OpenCode Go/DeepSeek API directly, it smells of a quantized down provider is mixed in.
OpenCode Go is becoming less of a good deal by the month. I pretty much only use it for mimo 2.5 pro now, and everything else is either ollama or openrouter.
Not in my experience. Tasks that would normally cost $0.08 on DSV4-Flash have cost me $0.30+ on GLM-5.3-Flash. These costs are after Deepseek's recent increase. Also GLM-5.3-Flash is so slow compared to DSV4-Flash. I would be fine with GLM-5.3-Flash if it was cheaper and at the same speed as DSV4.
I use DSV4-Flash on Max through Deepseek's API. I have been using GLM-5.3-Flash on High through Openrouter which I thought had a 50% discount. I must be doing something wrong for the costs to be off this much.
I've been using it quite a bit too. My main complaint is that it can be really slow sometimes — like, really slow — and the speed feels pretty inconsistent.
How much usage do you find you get on these kinda models (I know the pricing changes a bit) compared to a $20 sub say for Google AI Pro in anti gravity?
I hate how difficult it is to compare prices when looking at subscriptions.
Would $20 in open router, using models like GLM get me more or less?
I think it'd get you less than a $20 sub to any of the big three. I've used it on OpenRouter and found it kind of expensive for the results, but that might change now that it's open weight and other providers can host it/compete with Z.ai. For the work I did with it, I would've rather used DeepSeek V4 Flash just because it's more economical and still gives good results IMO.
Z.ai does have their own subscription, but I haven't used it because their privacy policy was pretty buns last time I checked.
Yeah I used Ox alpha earlier this week when it was free and deepseek flash when it was free on opencode. Both were great. Z.ai’s subscription doesn’t look very good versus the others.
I burn through my current Google AI pro sub for the week in about 2 and a half days so wanted something extra to add to it, but don’t want to buy any expensive ultra plan. Flash models have got me about 98% of what I need, but quotas are still a bit low.
just tested (zai-org/GLM-5.3-Flash via together.ai) against latest DeepSeek-V4-Flash for a very specific task and thought i'd report here...
- price: DS4 wins... $0.0235 vs $0.0242 for ten tasks
- latency: GLM wins... 108s total against 154s
this is for a personal use-case where i'm detecting ads in a written transcript. sticking with ds4-flash for now since latency is not a critical factor
GLM 5.3 is probably the sweet spot open weights model if you want to go beyond deepseek flash or the new glm flash. I used it with pi and had a fairly good time, especially since it’s less touchy about cyber and whatnot than the US guys. It’s slightly behind Kimi in ability but it’s a lot easier to run it, I’d expect prices (and speed!) from third parties to be noticeably better.
Assuming you’re willing to drop a fat stack of cash on the upcoming Mac m5 ultra with 512 gb unified memory, you can even run it locally, quantized to 4 bit. Whether it’s even slightly reasonable, well, my wife would probably skin me alive but maybe yours is more understanding.
When we consider:
* LLM usage is new for the world
* Models are evolving quickly with high worldwide competition
* Hardware is evolving despite RAM shortages
Is investing a huge sum of money in equipment for local inference a wise use of money? Or are M5 Ultra and equivalently priced local inference hardware future-proof enough to be worth it relative to how the market is evolving? Maybe it’s all a question of what you’d spend otherwise on serverless or dedicated GPU spend…
It is absolutely not worth buying hardware to run models for purely (long term) cost reasons. For open weights models the economies of scale means the cloud beats local significantly and your payback time is like 10 years.
However there are other reasons (e.g. privacy) that might make it worth running locally for some people.
So far I don’t regret buying an M1 Max device with 32Gb of RAM. The models available for it keep getting better (running just about okay for interactive use) and 400 GB/s of bandwidth is still considered a lot.
The models are currently improving much faster than the hardware and this doesn’t seem to have plateaued yet.
Cool! I'm thinking about a local set up. What's your usual tokens/second rate?
That's basically the question I'm trying to answer.
If you're paying Anthropic or OpenAI to use their models, harness, governance, etc., I could see the local inference potentially coming out ahead. They're already starting to ratchet down what your money gets you on their platforms, and that can be expected to continue as the leaders of those companies continue to seek the road to the El Dorado that is being a trillionaire.*
If you're looking to get into the guts of AI development instead of having it handed to you by a provider, that's where it gets murky. I'm wanting to write some sort of agent that does things and get into making outputs consistent in the like, and I'm not sure whether to host something on GCP or buy an M5 Mac.
*Note: El Dorado is a mythical city and many people died trying to find it.
My colocation costs come up at £7k a year for 3u. 1xu in prime dc, 2x1u in another DC for backup
My rented H200 server comes at just over £120 a day, £3.2k~ a month. Minimum two year contract.
I don't want to, but I am having to say bye to my 2x1u.
Part of it is knowing that whatever sort of enshittification the cloud providers do, my local programming environment won’t ever be less effective than it is today locally. It’s the same reason my entire development stack from editor to compiler is open source. I don’t need to modify it today, but I always must retain the option to do so later.
There are several things I do in my life that only pay off in the event of a big disaster, like an extended internet outage, civil unrest, supply chain disruption, war, etc.
I like to be able to do the things I do even if offline for weeks.
I spent a lot of money for more flash in my iPad Pro so I can keep all of offline wikipedia and OSM in it, for example, along with tons of books. It’s sort of like being a digital prepper. (Being a prepper is a spectrum, from anyone who keeps food in their pantry to people building bunkers under their house - how much you invest is a personal prudence and threat modeling decision.)
Also, privacy. And when I got the Mac Studio the 512GB was only $15k, which is dirt cheap for that much VRAM.
I have just built an Epyc with 512gb DDR4 3200 RAM for a "reasonable" price and I'm hoping to have a setup with GLM as the architect and Qwen 27b/Next Flash as the implementer. This is 1/5 of the price of the Mac, but also probably 1/5 of the speed lol.
I’ll be very curious what you get with DDR4. I also almost went that way. I have an Epyc DDR 5 rig and the best I see is 10 tok/s. Caveat being that’s at Q8 and a 4090 doing pre fill so it could be pushed up.
The surprising thing for me is how much work you will need to cool the banks if you’re near your memory ceiling. My memory starts soft throttling at about 74C (dies may be hotter, that’s the bank temp) and will turn down speed to try to stay below 80.
Happy to send my llama.cpp config settings if you want it.
I was running one of the older llamas (3.1 I think?) at slow-ish (10-20 tok/sec at Q4?) but OK speeds on 12 year old DDR3 ECC Xeon machine
I have a dual epyc + 1TB RAM. I could push glm 5.2 to 7 tok/s CPU only.
Curious about that price, if you don't mind sharing a ballpark
Depending on which Epyc you got it might be slower than 1/5 of the speed.
Honestly I suspect neither of them will be performing terribly well but with DDR4 3200 RAM I wonder if you'll be counting tokens per second or seconds per token. I mean, you do at least get a lot of memory channels at least, compared to consumer PCs. I am curious to hear what performance you get, I feel there is not enough information out there on what different setups manage to eek out.
The fastest I was able to get my Threadripper 3960X + 2x 3090s + 256GB DDR4-3200 to run a 2-bit quant of GLM-5.2 was 8 TPS. I would expect to be in seconds-per-token territory for a pure-CPU 4-bit quant.
One thing I'd like to try is MoE offloading: I have 2x32 GiB of VRAM and 128 GiB of DDR5 running at 4800 MT/s (only 2 channels though). I've seen people post difficult to believe MoE offloading results albeit a decently long time ago with older models. Maybe there is a quant that would fit with MoE offloading?
That said, I am guessing my problem is not enough RAM - but this poor consumer platform struggles to do memory training with 128 GiB as it is.
Now I surely regret not having gotten Threadripper and 256 GiB of RAM in the before-times.
My measurement was with MoE offloading, but there's only so much you can keep on-GPU with a 200GB quant and 48GB of VRAM. It's hard to overcome the CPU/RAM bottleneck.
For what it's worth, all of my hardware was used; I think, all-in, I'm probably at around 3k-4k USD? Not cheap, but also not the worst for something relatively versatile.
Ah, I see - so MoE offloading is no savior. A shame but no surprise either.
With a 4-bit quant of GLM-5.2, I can get about 0.8-1.1 tok/s on an underclocked dual Xeon E5-2698 v4 with 512GiB of DDR4-2400. I think it was specifically a Q4_K_M quant. Of course, the time-to-first-token is absolutely atrocious.
Which is completely insane for a ten year old configuration.
One could also run it locally on a used dual xeon (or amd-equivalent) server with 512GB RAM, albeit slower, if you have a useful workflow for it that's like "take this day's efforts and run it through various analysis agents", combined with giving it one-shot tasks/modules to build overnight. You would want a place like a garage or basement to put the server because it'll be loud.
> "dual xeon"
Does inference make full use of the memory bandwidth in a NUMA system?
Memory throughput performance is better than many equivalently priced (talking about the cheaper end of the used/refurb market here) single socket systems because each CPU is talking to its own banks of RAM.
Using an example of llama-server on a 256GB (each CPU is attached to 128GB) system running qwen 3.8-flash-next Q8, which takes up about 188GB with the model and context, it's not that the inference engine 'knows' about NUMA, but if you use --load-mode mlock to load the whole thing into memory at once at the time llama-server launches, the kernel will distribute it relatively evenly across the memory connected to both CPUs.
You'd also likely spend far more in electricity than the API cost of processing the prompt(s)
yes, though for some uses, not sending data anywhere to third parties has its own value which is harder to measure.
Its reasoning leaves a lot to be desired :(
Though I appreciate how good it is at "solid" grunt work and at that price (in fact I am paying the grandfathered subscription price; mostly).
I am planning to let go for my Claude AI subscription which I now use only for "planning" and maybe use that via Open Router as PAYG (at to try how it ends up). But god glm is bad at "talking" and "responding" anything prose. Not only quality but it's almost impossible to tune it and make it let go of its habits and biases and enthusiasms which often result in too many too and fro.
So I sometimes wonder at what point that starts becoming the cost and mental hassle. Maybe it's not there for me yet.
Well if you did get the m5 ultra could you obliterate the guardrails and then your wife can ask it pertinent but unsafe questions about how to punish you. Seems doable.
"My wife would skin me"
Oh please, as though significant numbers of tech workers on this website are not buying LV, Gucci, and yes Hermes for their pampered wives.
My wife can have an opinion on my tech purchases when I get an opinion on her 20,000$ bag addiction.
And the worst part is both of you should unironically accept these purchases, as somehow luxury bags, high end hardware, AND rolex's often out-pace inflation and are objectively good stores of value in a world of rapidly increasing income/wealth inequality.
You are allowed to have an opinion on her 20k bag addiction
Not every tech worker is making top-tier US salaries. For some (I suspect not few) people on HN that $20,000 Mac is almost a year's salary.
and even if you were making such a salary, the quesiton of if the investment on hardware to run llm's locally is still a big if, its OK if you buy the HW cause you'll use it and you get the extra capability as a nice extra, but doesnt make sense to spend so much when you could just get 200$ subs with almost infinite SOTA tokens a month etc (if you dont need the local/privacy aspects of it)
I'd like to ask Sam Altman if he still thinks that it's too dangerous to publish GPT-3. I mean, no one would use it, but what is his reasoning for not publishing it now, in 2026?
There are risks associated with releasing historical proprietary models that were not designed for open release:
- It is trivial to extract samples of the training data that was used, which can bolster existing lawsuits/foster new ones.
- Older models are not as safety-hardened, so it is easier to coax unsafe behaviour out of them, which is a PR risk.
- It may be possible to divulge proprietary secrets from the model (e.g. architectural details that may still be relevant).
For these reasons, and more, it's unlikely that GPT-3/similar models will be released until these concerns are no longer relevant (e.g. when they become a purely historic concern, similar to the open-sourcing of other proprietary software from decades ago).
I think it would be an important historical document as well. We are potentially looking at the dawn of AGI and one of the most important models ever created. Each model is also a kind of ultimate time capsule, containing a snapshot of the entire human collective mind. If you wanted to ask a 2002 person what they thought about future historical events you can just ask them directly.
> If you wanted to ask a 2002 person what they thought about future historical events you can just ask them directly.
The weights arent the truth tho, maybe a timecapsule-vhs but i wouldnt trust llm weights more than more hardcore deterministic media that might get preserved to infer facts from an era.
The companies doing the training are becoming the "winners" that are "rewriting history" as they train their models.
There’s not such a straightforward relationship between safety and model sis.
According to the book The Thinking Game, lower quality models at that time were considered less safe, because they could be easily tricked into doing harmful stuff. In the book, Dario (of Anthropic) was the head of safety at openAI and was responsible for pushing for 10x scaling in training to make the models safer.
It does make sense, a smart model is going to be way harder to jailbreak into telling me how to synthesize anthrax (or something).
> According to the book The Thinking Game...
According to me, this is nonsense.
I mean, none of this is controversial from a historical perspective, people did think that way. Whether they were right about us is another matter.
>but what is his reasoning for not publishing it now, in 2026?
What's the point of publishing it when it'll likely be outclassed by gpt-oss?
Maybe I'm reading too much between the lines, but I suspect the reason is to rub his nose in the duplicity or naivety depending on how generous you're feeling. Publishing the model would be a confession that he was wrong.
AI policy is being shaped somewhat by the things Sam and Dario say. So even if you're not feeling vindictive, it's probably good to keep a track record of the previous things they have said as a Bayesian prior. People who don't know better listen to these people, and maybe they shouldn't.
Wasn't the whole point of not releasing because they wanted to normalize the process/concerns for future actually dangerous releases?
I think the release of kimi k3 is definitely arguably dangerous, we're already seeing consequences of elite-tier cyberoffense capabilities.
If you want to dunk on sam for "it's too dangerous to publish GPT-3", you hardly need the release of gpt-3 to prove your point. All the other open model releases already provide pretty good evidence. Moreover the fact that the model wasn't release hardly points to the fact that he wanted to save face or whatever. Most AI labs don't release their old proprietary models, so the fact that gpt-3 wasn't released tells us very little.
Thank you, I couldn't have said that in a better way.
They already publish gpt-oss which is several generations better than gpt-3
GPT-3 is a different model than gpt-oss and is therefore not an answer to the question.
I cannot stand using gpt-oss, but I miss some of the creative spark of GPT-3 davinci dearly.
I’ve found a very similar creative spark with Gemma4 base models. You’ll have to do old school prompting, but it’s kinda fun too.
At release, GPT-OSS was arguably a few generations behind the open frontier.
No? They were the frontier, or near it, at the time of release: https://artificialanalysis.ai/models/releases/gpt-oss-120b
I previously posted that DS4Flash was _good_ but not _great_ on two DGX Sparks, but I have to say that GLM-5.3 is pretty amazing. It's been able to tackle all the random hard problems I've thrown at it and it has the intuition that DS4Flash seems to lack.
We're nowhere near a Fable-class model IMO, but things are going to get interesting in this next year.
I assume that's about 5.3 Flash, not full?
What quant are you running and tps?
NVFP4 ~20-30tps (MTP + vision, no dflash2).
I've been using it more and more. Feels like Opus 4.8, in the best possible way.
I'm starting to think Opus 4.8 is significantly smaller than most people assume.
If it's significantly larger than GLM 5.3 (I've heard some insane guesstimates out there like upwards of 5T params or more), that would prove rather embarrassing for Anthropic.
I hear the argument here, but isn't it possible it has dramatically more knowledge and when you get outside the common cases many of us use it for, it'll have completely different capabilities?
I feel like most benchmarks cluster on a reasonably limited area of human knowledge
Sort of depends on how well the core reasoning works. It’s not a big effort to connect an LLM to a search provider.
You do pay for the tokens, but in theory on a smaller model each token is cheaper.
honestly using search isn't that great, you mostly get SEO slop, it usually won't help the model ask the right questions
Try Parallel.ai (no affiliation). Instead of keywords, the model writes objectives.
I think the Western labs are burning through funding and compute to maintain the lead at any cost, efficiency be damned.
It seems like there is tradeoff between model size and the need for tool use, which - in my mind - is quite costly in terms of time and tokens. More detailed world knowledge requires an exponential increase in model size, but most knowledge can be acquired ad hoc using search or database queries. This will fail for questions where the model lacks the knowledge to ask the right questions, but maybe this could be solved by a handful small inquiry models with different knowledge encoded in their weights?
Or calling into a full-knowledge model “I’m facing problem x, how do I ask myself the right questions?”
I should do that myself, come think of it.
You can't compare models released 6+ months apart. GLM 5.2 was same architecture as 5.3 and not nearly as good. Takes time to build frontier intelligence and distill down to smaller sizes.
It's not that GLM5.3 in full precision unquantized is any smaller, it's 141 * 5.4GB files at approx 770GB which is about the same size as 5.2.
Hold on.. the routed experts are in FP8 now? Previously they were in BF16. Nice, this shall cut my download time by half!
> that would prove rather embarrassing for Anthropic
Not really, in that you just work with different constraints.
Anthropic and US labs in general has maybe 100s to 1000s of GPUs per person to experiment. Zai and Chinese labs in general have 1-10.
The priorities are different.
Do you use it to write HTML/CSS? Javascript? C++? There's a huge difference in ways people use models and if you are not specific about it then your comment means nothing, unfortunately.
I really like how it doesn't have that Claude talk. It just does the thing without Claude's "load-bearing honesty." It's probably my favorite model to interact with, even if it isn't the best or most reliable.
My second favorite model by now is GLM 5.3 flash which is very capable of day to day task. I use it as the main model and GLM 5.3 for task that is more complex
Can you give ant more details how you are you using it? Which harness / service / what you’re building with it etc?
z.ai coder plan, both in opencode and direct API access. I use it for my side projects like https://github.com/scosman/Biscotti (on-device meeting transcription and summaries).
> in the best possible way
You implying its better than opus 5?
I have been only using GLM models since last December and have had the best experience without any drama about tokens and geopolitical restrictions. The quality has been great and I am doing more and more with the latest 5.3 and am really excited that consumer hardware will develop in the next few years where I can run these at home.
h/t to DeepInfra for being the first 3rd party provider for it on OpenRouter (https://openrouter.ai/z-ai/glm-5.3?endpoint=b711bea7-3994-49...).
their cache hit rate is 67%. In comparison the provider with the highest hit rate is at 95%.
on my TrustedRouter:
z-ai/glm-5.3: also Z.ai, Novita, Atlas Cloud, IO.NET
I have seen you advertise your website a few times. I like the idea of not having to trust the router, so I took some time out of my day to critique your website: https://files.catbox.moe/v68cf7.png
My visit to your website went like this:
1. Visit models page
2. Try to find GLM-5.3-Flash (which is among the ~5 models that 90% of people currently care about)
3. Give up scrolling (which would have taken OVER 50 SCROLLS!!!) and use Ctrl + F
4. Try to find input/output/cached price
5. Scroll all the way up to find out which column is what
6. Notice that output price is cut off
7. Notice that the scroll bar is over 100 scrolls further down the page
8. Use Shift + Wheel to scroll horizontally (most visitors probably won't know this trick)
9. Notice that cached price is missing
10. Conclude that this is probably not a serious offering and bounce
There are probably more issues later on, but this is how far I got.
I would suggest you to:
- Deslopify all pages that a user may visit before conversion
- List important models first (see OpenRouter rankings)
- Move the most important information (model name/input/output/cached price) to the left
- Disaggregate the prices per provider (maybe subtables per model? not sure)
- Measure cache hit rate and compute effective price per provider (see OpenRouter)
(- Optional: Fix the broken link on your HN profile page. Currently, the only way to get from this comment to your website is a search engine.)
It's also now live on Ollama Cloud as of a couple minutes ago
Have you guys been having a good experience with OpenRouter? I tried it out recently with Claude, and it cached no tokens, charging me $200 for one conversation of 11 messages.
I tried using deepseek v4 flash with OpenRouter. It switches between providers too eagerly which resets the cache. Then, each provider begins to rate limit me for providing so many uncached tokens, so it just keeps on switching providers. I'm paying for every token... why rate limit me? It was unusable compared to just using the official Deepseek provider which has a much better cache rate.
I just checked because I was a bit paranoid, but I have a 96.6% cache hit rate for GPT-5.6 Luna and 96.8% for Opus 5.
Hm, thanks, it must have been some OpenWebUI bug, thank you.
Does this mean it'll be on Bedrock soon? I hear great things about this model but I want AWS data handling practices...
A stake through Amodei's heart.
how feasible is it build a SOTA specialized model for some use case e.g. deal sourcing by using this as pre-trained model or a LORA or similar pattern on top? Gonna shoot my shot at a billion dollar business
Looking at HF, it looks like the unquantized version is half the size of glm-5.2 756 GB vs 1.51 TB. I wonder how they were able to optimize it this well
FP8 experts not BF16.
FP8 vs BF16. But yeah, nice job.
Is it possible to fine tune this model and unlock / extend its cybersecurity capabilities? I'm scared that maybe we are not ready for an open-weight model with high cybersecurity skills.
kimi already exists and yes it is more than possible. it's coming
brace yourself
GLM-5.3-Flash is actually cheaper than deepseek and better than deepseek but no one is talking about yet :)
It's actually slightly more expensive ($0.50 vs $0.48), but there's a temporary 50% discount.
I've seen dozens of conversations about it in last 24 hours, and every major inference provided added in first 24 hours. I think it's gaining plenty of traction.
It's interesting that OpenCode Go is treating it as 2x more expensive than DeepSeek Flash, even factoring in the 50% discount
OpenCode Go is probably using quantized down DS4Flash. They outsourced to 3th party providers to keep the cost down, and being able to provide that $30 value (instead of the initial $60 > $15).
We saw the same issue with GLM 5.2 when they still published publicly who the providers are on their website. Most ran FP8 but one was doing FP4, so you had this issue where one moment you had the better FP8 and another session you had the FP4 provider.
You can check the internet archive, it was in the FAQ part before they hide/removed it. So if you looked up the providers, and the published quants, yea, ...
Given that a lot of complaints are coming from people that felt OpenCode Go Flash feel like a step down compared to old OpenCode Go/DeepSeek API directly, it smells of a quantized down provider is mixed in.
OpenCode Go is becoming less of a good deal by the month. I pretty much only use it for mimo 2.5 pro now, and everything else is either ollama or openrouter.
Go has API pricing + this weird scaling of how much is it worth. Some models get $60 of usage, some $30 and some $15 etc.
Not in my experience. Tasks that would normally cost $0.08 on DSV4-Flash have cost me $0.30+ on GLM-5.3-Flash. These costs are after Deepseek's recent increase. Also GLM-5.3-Flash is so slow compared to DSV4-Flash. I would be fine with GLM-5.3-Flash if it was cheaper and at the same speed as DSV4.
I use DSV4-Flash on Max through Deepseek's API. I have been using GLM-5.3-Flash on High through Openrouter which I thought had a 50% discount. I must be doing something wrong for the costs to be off this much.
I've been using it quite a bit too. My main complaint is that it can be really slow sometimes — like, really slow — and the speed feels pretty inconsistent.
z.ai is using all Chinese hardware for flash: https://thenewstack.io/glm-5-3-flash-chinese-chips/
There are other providers with much faster inference, like BaseTen at >100t/s: https://openrouter.ai/z-ai/glm-5.3-flash#performance
How do I find out where the openrouter model providers' servers are located?
If you click on the provider name, the panel that pops up shows a "Region" value. Not every provider lists their region, however.
I think the region is just the HQ of the provider. So z.ai's region is Singapore but it's quite likely that their servers are actually in China
It’s cheaper sure, but it’s very slow. It’s not a drop in replacement
I think we don't have a good draft model for better speculative decoding yet (e.g. DFlash 2). Once we do, it will be faster.
It very well could be faster, but right now it isn’t.
It is a slow for me through z.ai; it does not feel 'flash' at all. But then neither did the new DS Flash. I think they were getting hammered.
Seeing the price, I am probably just going to keep using GLM-5.2 until 5.3 gets cheaper.
isn't GLM 5.3 flash better than GLM 5.2 overall?
How much usage do you find you get on these kinda models (I know the pricing changes a bit) compared to a $20 sub say for Google AI Pro in anti gravity?
I hate how difficult it is to compare prices when looking at subscriptions.
Would $20 in open router, using models like GLM get me more or less?
I think it'd get you less than a $20 sub to any of the big three. I've used it on OpenRouter and found it kind of expensive for the results, but that might change now that it's open weight and other providers can host it/compete with Z.ai. For the work I did with it, I would've rather used DeepSeek V4 Flash just because it's more economical and still gives good results IMO.
Z.ai does have their own subscription, but I haven't used it because their privacy policy was pretty buns last time I checked.
> but I haven't used it because their privacy policy was pretty buns last time I checked.
What did you find objectionable? I looked at it when I subscribed almost a year ago and I was fine with it (e.g. they don't train on your API inputs).
Yeah I used Ox alpha earlier this week when it was free and deepseek flash when it was free on opencode. Both were great. Z.ai’s subscription doesn’t look very good versus the others.
I burn through my current Google AI pro sub for the week in about 2 and a half days so wanted something extra to add to it, but don’t want to buy any expensive ultra plan. Flash models have got me about 98% of what I need, but quotas are still a bit low.
glm 5.3 flash is the best model I have ever used
just tested (zai-org/GLM-5.3-Flash via together.ai) against latest DeepSeek-V4-Flash for a very specific task and thought i'd report here...
- price: DS4 wins... $0.0235 vs $0.0242 for ten tasks
- latency: GLM wins... 108s total against 154s
this is for a personal use-case where i'm detecting ads in a written transcript. sticking with ds4-flash for now since latency is not a critical factor