I feel like I must have plateued and don't know what to do next to level up. I'm currently on the $100/month codex plan and it seems fine using 5.5-xhigh all the time. I think of what to do next, have a chat session to determine exactly what to ask for up to the point of being ready to implement, and then codex churns on a commit-sized task whereupon I briefly check it on my local dev server. If necessary I ask for a change. Then I ask it to commit and recommend the next step based off the spec. Oftentimes I have to "approve" an out-of-sandbox request anyway.
I haven't found anything that requires running all night. I could tell it to one-shot a big plan but given how often I realize I want an intermediary thing to be slightly different it seems like a waste of effort.
I'm guessing the next thing I should probably look into is some sort of machine vm I can tunnel my codex-gui requests to so I don't have to deal with the sandbox approvals (I don't want to give it "dangerous" access to my entire mac).
I don't understand what people are doing with their side projects that is leading them to churn through tokens so quickly, to the point of requiring two $200/month subscriptions and a bunch of token charges besides.
I’ve watched a bunch of layman videos where they create stuff with AI, these people burning through 12 hour tasks are literally not reading the output or understanding what it’s doing. Like they’ll ask for a program, and then right after it’s been created they ask the AI how to run it. Then when there’s a bug, they ask the AI what went wrong, or scrap the entire thing and switch model/harness and try again.
Amazingly, there are people out there (apart from creators), that work that way in their day-to-day job. I had the pleasure to work with such a person. After several months, he got removed from the position. He left a mess that hasn't been cleaned up completely to this point.
It won’t be long till employers get wise to this stuff, they just need to burned a couple of times.
It seems AI is good, great even at many things. But it doesn’t seem like it’s going to change the world as much as some people believe it will. And if it does it’s going to take time
I have downgraded my Claude to the $20 one, and basically only use it for the web chat right now. For coding, I use DeepSeek @API Rates configured in Claude Code. I have spent around $4.8 for 320,000,000 tokens. I always felt like i was not using Claude plan, that i had to have the LLM working on something all the time to justify the price. Now with DeepSeek i don't think about it anymore. I don't feel bad when not using the subscription anymore, and i don't worry about limits as i just pay more. Where i really felt this was on running things in parallel as there are no hourly limits anymore!
I’m using $200 a month Codex working on a game for my kids for fun and curiosity since I’m a dev, I’ve played games, but I’ve never done dev for games. and have all night tasks but mostly they’re “spend time tending to and adding stuff to my 3D asset pipeline”. My RTX 5090 runs Trellis2 -> ultrashapes -> Trellis2 -> wiring up rigging and setting up animations.
But like 99% of that task is just Codex waiting for the output. So it’ll run for 12 hours but mostly it’s just setting lots of sleeps. I haven’t gotten close to running out of tokens. The $100 a month codex I hit usage limitations almost immediately, about 3 days in of working like crazy with 10 agents going at once, mostly coding an asset pipeline, I ran into my weekly limit and upgraded. So with the $200 a month plan at 4x more credits I haven’t hit any walls at all and can absolutely cook.
I have been on $100/mo claude and it has been churning out quite good software for months now. like i estimate what would have taken me three ish years, assuming i didn't burn out from failure (i would have). i only hit limits when i double fisted claude with my main project and my side project. just the other day i noticed i had been stuck on 4.5 because i failed to update the npm package.
Patches to existing sizable codebases and reverse engineering binaries both can run a long time and use a lot of tokens without wandering off into the weeds.
I'm on $100 Claude. I have a setup with bespoke local services that mitigates some high token consumption scenarios with local LAN services. I screen mcp's and hooks for cache poisoning. I run 100% on Opus with max effort, and never came close to hitting 5 hour or weekly limits before the Fable release. I am in Claude Code at least 20hrs a week.
I see people just completely wasting tokens with ridiculous setups, 100% hitting cache misses as well as dumping huge files into context all the time.
Just learn how these things work, or pay the price I guess.
Same boat here. I’m able to get a lot done on CC at $100/mo and feel like I’m not being creative or productive enough somehow when I hear of people blowing past that in a day.
I can get away with the $20/month plans, even on extra high, when I'm the one primarily calling the shots and writing code. I use the LLM for intelligent codebase searches, reviewing my code, refactoring that I direct, and scaffolding simple features for me to pick up.
I have also tried having the LLM write everything for non-critical, temporary projects that I thought were simple. It's amazing to watch how much the code explodes into spaghetti if you're not intervening. Every day the codebase would explode with more stuff: Extra abstractions, more arbitrary tests, harnesses to support those tests, leftover cruft that the AI stopped noticing and left in the codebase.
As it all accumulates, each new change request to the LLM turns into a journey through the mess it has created for itself. Even changes that I thought would be small could turn into 30 minutes of churning different tests that were added in the past and intersected with the change in an unexpected way.
So you can try to contain it by running more agents and spending more tokens to try to do clean-up passes and architecture rework, but it's a constant battle. You also have to be in the loop frequently to stop them when they're on a mission to do something you didn't intend, don't want, or recognize as a bad idea. If you're trying to get it done in the background, then you really just let it go and hope you can spot the problems later and have the LLM fix them.
None of these problems show up when you're doing these amazing YouTube or Twitter demo app posts where you go from 0 to something that looks like an MVP in a short clip. The LLMs are amazing at doing that. It's all of the problems that arise with maintaining it afterward that will start consuming tokens at rates you didn't think were possible.
I think this is also why we don't see many true vibe-coded production apps: It's really easy to get something started with an LLM and even shipped to production, but the technical debt load is so staggering after a while that your "agents" are spending hours trying to unwrap everything to do every single feature request.
Luckily I needed a new laptop and I bought an M1 Max secondhand from a friend quite cheaply because it was fast enough to recompile something else I am interested in.
So for me, there is no additional hardware cost; it was acquired in replacement.
I run the AI models at home on this kit because I want to; I'll use openrouter if I need to.
I accept the economics of this article are right. But I feel so incredibly sad about this outcome that we're now just to be people caretaking machines that do the job we loved that actually I am not sure that exercising this nuance is going to matter in the long term.
It turns out it is a mistake I have made in my life — now really unfixable because I am a bit too old — to believe that I will always find enough fulfilment in my work to offset the absence of personal fulfilment elsewhere; I have always enjoyed being able to help people directly by doing a thing I love and I am good at, and that has kept away the sadness of finding it difficult to build a conventional family life to enjoy.
I assumed I would always find some new way to find that enjoyment, but even the slim enjoyment from being able to explore this stuff on my own kit in my own terms will not be enough if the pendulum does not swing back towards human effort.
It is a dismal world we have made for ourselves. Lately I have found myself dreading growing too much older in it.
You sound awesome. Just venting? (b/c curious if friends can fill your heart abundantly, & we know we're never too old to make new friends!)
> dreading
Even avoiding political headlines (OK, at least articles), plenty of cause for dread, so I keep re-focusing to avoid despair. Easier said than done innit!
Can't kill my hope for the future though. One day, all the good stuff shall prevail (morality, intelligence, love & kindness)... maybe not permanently, but a Star Trek future is there somewhere (& they had their troubles but it wouldn't be a dreadful situation overall). Sharing with you in case it's even slightly contagious!
Also, I would anticipate at least a 5 year lifespan for a current generation card. The 3090 is still respectable simply because it has 24GB of RAM which, for years, has been the limiting factor for ML at home. If you got a 6000, sure it’s going to cost 7-8k, but the resale value is likely to be very good. Even the 3090 is 50%+ of RRP still. And if you’re not doing LLMs, it’s an interesting value proposition for “classic” CNN vision model training. You can fit enormous batch sizes on 96 GB. The biggest reason to upgrade is perf/watt has about doubled (eg 4000 pro Blackwell is half the 3090 for similar).
People tend to assume the capex is thrown away but as we’ve seen with RAM, don’t be so sure you won’t be able flip it if you need to.
If you have solar, it is not, because you have battery and equipment degradation from cycle charging, c’mon man…
I would agree with you if you said it was vastly cheaper overall (with the initial equipment investment amortized over time) compared to The Power Company.
In many states, even if you are generating electricity and selling it back to the power company, they still gonna charge you normal rates of usage because greed.
If you go off grid, you have bigger things to worry about than how to power your AI cluster. It’s manageable enough if you have land but that’s in scarce supply.
> if you have solar, it is not, because you have battery and equipment degradation from cycle charging, c’mon man…
no, the rate of that is pretty independent of use. unless you live in a place where selling energy back rules are designed to screw the solar owner (California)
There's actually an interesting thought experiment here: if it takes you a full day to build something that AI would otherwise build in a day, do you end up using more power, or less? What is the break-even point, purely from a power consumption perspective?
If an identical task takes a day on both sides, then the human route uses less energy, surely.
Brains are thousands or maybe even millions of times more fuel-efficient than computers and you are alive for the whole day either way, right? You probably eat about the same even.
The reason executives think AI is more efficient is that it more space efficient than a human and doesn't demand to be paid or work only a set number of hours. Everything with computing is more efficient if you resent having to give money to other humans. If they could just not have you be alive when they don't need you, it'd possibly be different.
Even though I think at a typical British freelance rate and a truly unsubsidised token price, the AI is possibly more expensive than me. And as a freelancer, from their perspective I really am not alive until they need me. (This is what it often feels like)
The reality is the human and the AI aren't used to build the same things anyway so it's a comparison you can't really make.
Brains are efficient, but civilized humans aren't. In the USA, adults consume at a rate of about 10kW -- only 1-2% of that being the human's metabolism, the rest being HVAC, electrical devices, etc.
For comparison, a modern frontier model like Gemini 3.5 Pro consumes about 15kW -- so only about 1.5x the fully loaded human. In an 8h workday, that model would crank through ~80M tokens (~$5k at API prices). That's ~4 major refactors of a 10k LOC codebase, so probably not a very realistic comparison to a single human dev.
I think a more useful comparison, based on my experience, is that an engineer with AI support can get one 8h day's worth of unassisted work done in 1h. So, the 25 kWh consumed during collaboration (conservatively assuming I keep the GPU hot for the whole hour) frees up the remaining 70 kWh I'll draw down for the day to be spent in some other way.
Studies on grandmaster chess players indicate that at most you burn 10% more calories when engaged in deep thought than when you're at rest. So the energy "attributable" to an hour of knowledge work is like 10 calories (average sedentary calorie burn is like 80-100 per hour; add a max of 10% for the thinking gets you 8-10 calories). A pound of potatoes is like a buck and is about 320 calories. So you're looking at like 3 cents an hour at most to cover that energy burn. It's definitely even less; I certainly don't think as hard as a grandmaster chess player.
Then, assume power costs 20 cents per kilowatt hour (US avwrage) To match the human 3 cents per hour, you need an average of 150 watts of power drawn per hour. That's in the range of a budget graphics card, but not much past there.
However, if you sleep instead of sitting around, you can probably make AI cost competitive. Sleeping drops your metabolic rate by more, and lying down in bed (as opposed to sitting) also reduces calorie burn. Combined, you can reduce your burn by like 30 calories an hour. At the new 9 cents per hour human cost, you can afford to run a higher end graphics card at ~450 watts per hour. That puts you in RTX 3090 range.
Speaking personally: yes. That's literally what I'm planning to do this afternoon because it's noon and I'm already done with the coding tasks I had on my plate today.
Luckily the future is absolutely going to be that star trek one where technological abundance means we are all wealthy and have free time to develop personally, and not the future where all the money bubbles up into the hands of a thin-skinned malignant narcissist who wants to play with launching rockets and provoking racial violence /s
I'm assuming that you need to feed the human being (i.e. you) regardless of whether you use that human being for writing code or not. So, by this metric, there is simply no breaking even point. The cost of human + AI is always going to be higher than the cost of human.
What are people doing at home? I have like 5 different apps I code on the $20/month Claude plan and like sure I can hit rate limits but - What are people doing to burn through $3k in tokens?
Same for me. $20/mo is just fine and I use it to code daily.
I suspect the people that burn through tokens have several subagents and 50 skills loaded and 40 MCP tools. All those load up the context on every single turn.
YMMV but automations eat through the $100-$200 plans, which burn thousands in tokens alone.
I have hourly automations for root cause analysis on customer support issues, daily automations for eg log analysis, weekly & monthly automations for SEO/miscellaneous KPI tracking & actioning.
I will say, when I was building side projects that were 1) fairly well defined in scope and 2) without users/need for automations it was much easier to stay under $20/mo plan limits. Now I regularly hit weekly limits and need multiple Max plans
I invested about $4,000 in an NVIDIA DGX Spark several months ago. 128 GB of unified RAM, and the NVIDIA GB10 chip. With the RAM, the several CPU cores, and the 4 TB NVMe SSD, it's a very capable ARM64 Linux computer even without the GPU, and so far I've mostly been using it as such. But I wonder, what's the most capable model, specifically for coding, that can run well on that hardware?
>Around $400 a month of plans buys roughly $2800 of API usage at list prices, which is a real bargain right up until you hit the ceiling. The plans are metered, and any large AI native workflow will chew through the included tokens fast
I don't think that's true at all. I'm doing 8-12 PRs a week at work, all primarily Claude Code, and the usage at API billing has never broken $500/mo.
I find just going via Deepseek's platform API directly, using their V4 flash model, and hooking into a harness like Opencode more than acceptable. Think I've spent maybe $10 over a couple of weeks.
I did explore self-hosting models but hardware right now is just too expensive.
Directly at DeepSeek? It was my understanding (but I didn't check) that some other AI operators were providing (some of?) DeepSeek's model for cheaper prices.
Still, that's interesting. What do you get for that price? Only coding, or also e.g. image generation?
For me, investing in hardware seems to be the way to go.
I learned coding nearly 24 years ago and still learning new stuff all the time. At no point in time I had to rely on a subscription model to learn and do new stuff.
If LLM and agents are the default tools for coding and building software, at least for next few years, it seems like a no-brainer to invest $2000-3000 on hardware, like a Halo Strix PC.
I wondered if there might be a no brainer "free" option on discarded hardware.
I have a GTX1080ti which i think is circa 2018, it's unused, more than paid for itself over the years, owes me nothing at this point so the hardware is free.
It runs Gemma e4b multimodal, qwen 3.5 8b or the qwen 4b embeddings models well enough (40+ t/s for the LLMs).
The machine consumes 350 watts at the wall when under load (3 watts when sleeping, 80w at idle). Electricity costs me £0.035GBP/kwh which is cheap for the UK (load shifting via house battery).
144k output tokens for around 1pence (and takes an hour to do that in theory).
It's only JUST cheaper to use than the far more capable deepseek v4 flash model despite the free hardware and ~10x cheaper than normal electricity.
Yes and no. Hardware does lock you in. Granted, I am happy with my 128gb of shared memory, but I am mildly concerned that it actually is more expensive now than when I bought mine. It does not bode well for the future; not when combined with recent WH admin moves on Anthropic and the reality that next batch of good models may require more than 128gb to run well.
edit: I am not dismissing local. I am one such user ( though I have subs too ), but one has to be clear eyed about the trade-offs.
$3k isn't getting you frontier model capability. It's barely getting you any capability if that's split into buying an entire PC rather than just GPUs.
With you here. I'm using my cheapo 16gig vram card I picked up a year or so ago, and I'm like -- yes, I percieve that you can pay for way more tokens per second that I can do at home.
But that feels like measuring productivity in lines of code. For what I'm doing, I'm not seeing the benefit in any subscription.
Sure, I can't one-prompt a whole new boring CRUD app, but oh well.
> The upfront cost is steep and the models you can actually run at home are weaker than what the frontier labs ship, so this only pays off if you can keep the rig busy with long running tasks where a slower, cheaper model grinds away overnight. Most people can’t keep a home machine that loaded, and the hardware you buy today may look like a bad bet in a year.
Oh, so this is not a post about AI coding at home. It's about vibe coding at home.
There's a lot I disagree with in this post, but I'm posting this from a home computer with 64 GB of RAM and no GPU. I do lots of AI coding while spending very little money. I run Gemma 4 26b (mixture of experts) and Qwen 3 coder with Ollama. I use Github Copilot code completions. I use the Gemini and Mistral API free tiers. I have a Gemini paid API account. It's now prepaid, so you don't have to worry about an accidental $1000 bill. You can do a lot of things with Gemini Flash Lite 3.1.
None of this is burning through tokens to create an expensive blob of spaghetti code, but it does qualify as AI coding.
My sentiments too. I'm using Qwen 3.6 35B A3B on a machine with 64Gb ram and a 24GB 5090 (an Alienware 16 Area51 I bought, serendipitously, about 15 seconds before the idiots preordered all computers for the next 3 years and ruined everything).
You can't "slop cannon" vibe code with it, but this is personal code I want to not be spaghetti, so I'm not trying to vibe code. I just want to get instant retrieval of all stack overflow and reddit posts in a chat box, and for it to be able to spare me the physical pain of actually having to type out typescript code (I am a BE dev with negative patience for all frontend) and fuck around endlessly debugging obscure docker problems (I like docker, but, no patience for it having annoying problems and endless quirks). And this model does that really well.
Can I run something comparable to Opus 4.6 locally yet? I keep hearing conflicting things. If I can spend 10k to do that I would cancel my subscription. The problem is I don’t wanna spend the money to find out myself.
If you want frontier-level, the economically reasonable option is OpenRouter or a direct sub to frontier-of-your-choice.
The reality is that they do not offer configurations that would allow a consumer to run that much VRAM on a single setup to protect datacenter margins. Apple used to, and they stopped, those devices are going for ~$20k+ each on ebay now.
You can get very, very capable models on a 3090/4090/5090/6000 series card. But if you want 'frontier level' you are investing ~22k at a bare minimum if you go new. Used you can probably build your own server for much cheaper up-front cost but it's likely going to be 4-6x+ electricity usage.
There are also significant economies of scale (namely: utilization and batching), which tend to make inference on a shared server more economical even after the operator takes a cut.
You can use batching on consumer hardware, it just requires a KV-cache efficient model (or short context only) and keeping multiple inference flows running in parallel. This is most useful in combination with streamed inference, since the compute intensity of decode with those newer KV-compressed models is high enough that you have limited compute headroom when running at the speed of RAM.
I truly think by 2028 we'll have integrated chip systems that'll be able to run opus 4.8 level models at ~500 watts at acceptable performance. Honestly I think now is the worst time to invest in AI hardware. Get your harness ready and processes perfected with hosted models, and wait a few years to buy hardware to transition to running models locally
Trying Taalas is almost scary, there is something unsettling with that speed! Even with that small model, because of the speed, you could run hundreds of sample runs in a second, and pick from the best.
the difference is that Putin's hand was forced by age, (possibly) illness, and the last several decades of how he chose to run his country. Putin's power base is a relatively small group of elites and oligarchs who would happily snuff out the man who pushes them out of windows if they get too uppity, if they were given the chance. He needed the cover of war to maintain the fiction of his type of strongman "only I can save us" leadership.
Xi's power base is the simple fact that his leadership has transformed China into the #2, and now because of Trump possibly soon the #1 world superpower. He has also acted aggressively in the last decade to find and remove corruption and prevent individuals from accumulating the kind of wealth and influence that could threaten his power from outside official Party channels. Of course, as I'm not Chinese myself, I have no clue what the internals of Party politics actually look like. But as an outside observer it seems clear that Xi et. al. do not actually need Taiwan for anything other than national pride. They know the US would go to the mat to protect it as TSMC is extremely vital to US military power. And since China cannot compete in that arena and has too much to lose, they instead have focused on weakening the US from within, quite successfully of late.
By the time China finally takes Taiwan it will be with little fanfare and little consequence - they won't touch it until the US either has lost its military capabilities, or the US has its own internal chip industry. Anything else is an existential risk for the coastal cities that are China's entire economic advantage.
Some benchmarks have shown Kimi K2.6 within error-bar distance of Opus 4.6, and you can run it on eight RTX6000s. Right now it's not possible to set up a machine like that from scratch for less than $100K... but right now it's also hard to put a price on autonomy.
You need a lot less than that if you're willing to stream the model from SSD. At that point, the best machine is probably a cheap old-gen HEDT with lots of PCIe lanes to attach cheap NVMe storage to, so as to stream the model at reasonable speed. That's expensive but not $100k expensive!
Best you could do is connect two Mac Studio M3 Ultra 512G RAM each with Thunderbolt. Then theoretically you can run frontier Chinese models (but not Deepseek v4 Pro yet). That would be about $20k.
But - good luck finding them. Apple discontinued the model a few months ago. And more recently, even 256G model was discontinued. Big AI really really does not want people to get off their needle.
DeepSeek V4 Pro is ~800GB total at native quantization (1.6T params with most being 4-bit) so it can run on the hardware you mentioned. There is also a 2-bit version that will run on a single 512GB machine. SSD streaming also makes lower-end hardware viable to at least test the model, if not quite run it usefully.
> Do that well and you can build what a team of twenty engineers would put out in a month for around a thousand dollars.
What does this look like after 6-12 months? Like, how much code are you trying to write total?
Maybe it just doesn’t click in my mind, but sometimes I wonder about how much work people are trying to do and how they actually have enough to get done so quickly in such a short amount of time.
They prefer to work harder and not smarter. Forever hill climbing to nowhere.
I've never worked on a complicated codebase that started out that way until the rest of the business concerns and office politics came into effect. People may not like it, but the bureaucracy is far and away more valuable than the core functionality.
Mature codebases are years of people thinking of all the possible gotchas while solving their acute pain points. This is not fluff, but the living and breathing part of it. Without that code, it's just a machine barely doing stuff in the most obtuse ways possible that nobody wants to pay for.
I would argue that they're putting LLMs to work on that finer detail stuff, but AI is still far too dumb. No, what they're doing is playing with their skinner box.
AI coding at home literally costs $100/month. I'm wondering where $400 is coming from? $100 is more than enough for "coding at home", IMO. I rarely face the limits, and when I do it's just a time for a quick walk anyway.
Man I’m using the $20/month sub and it works just fine for me. Granted, I have a family and house and lots of obligations so by the time I hit the limits some other task is due before I can return to coding. If I hit the limits before I have something else to do then I just code by hand or review what has been generated until I can use the agent again. Reviewing agent code is a good way to learn too, agents have shown me different approaches than what I would have done and they’re definitely worth thinking about. Also, fixing their mistakes has helped me write better prompting although being a team lead for half a decade has taught me how to specify what I want very clearly and cc gets it right most of the time haha
About interruptions, one thing AI assisted coding really helps with is coding with constant interruption. I can leave CC for half an hour and return then tell it I had to step away, catch me up, and proceed. This works well for me.
I think this is only going to become more relevant. I'm personally a $200/mo Claude Maxer and I know that the usage I'm getting on Opus 4.8 Max and (until they yoked it out from under me) Fable 5 is way, way more than what I'm paying them. At some point, this will turn usage-based and I will be hammered on it and probably forced to look at self-hosting. I think while the caps are there, even at $200, it's honestly not too bad if you're coding value into the market, but as soon as those caps come off for retail AI users, we're all going to have some tough choices to make.
I think someone could find some way to use the smaller local models to write code. Some kind of framework or harness or language or something. But not too many people are working on that because the big models are pretty cheap and a lot better.
Maybe one possible path(to make weaker models highly capable) is making the job of the llm as easy as possible.
I wonder if part of the solution is building/finding the right libraries, with the right documentation/language/API(one that plays well with LLM's) and maybe creating some synthetic data around them - to make it very easy for the llm.
And maybe there could be a business model around creating those libraries.
So in my limited experience: The smaller the model, the bigger the harness. The biggest issue becomes the context window. For big models you can kind of just give it bash access and let it run... while with the smaller ones you need to fully manage the context in each LLM call.
If you can ask the model for a specific function; with a spec design (typed languages help too) then the small models are great! I have had good progress with generating small python modules for example, but you need verification rounds to catch issues.
So test driven design + a good spec sheet + a very detailed todo.md (or even better if its todo.json because then the LLM does not need to manage it, you do from the harness) is your best bet for small models.
I think as well there might be "algorithms" that can work with local LLMs. With local LLMs there is a small context window, but not that much cost per token. So perhaps there is a way to do lots of small prompts that work in a sequence to produce a result.
Like perhaps you could produce 5 versions of a piece of code, and then compare them to choose the best.
Also if the local LLMs can call tools, maybe you can use static analysis tools to catch errors and try again in a loop or process of some sort.
There also might be certain languages that work better because those languages have better static checks.
I mean, this is what I'm doing. I'm guessing my process is very different because I'm holding the hand of the project way more along the way, but even that to me probably makes for a more enjoyable.
Which is to say, I might use AI to do an outline/organizational , but I'm prompting every chunk of code "one-by-one," (e.g. at about the "function" level) which still feels lightyears ahead of what I used to do.
I recently made an AI Agent and surprisingly coding with DeepSeek V4 Flash is quite cheap. It probably has to do with the aggressive prompt caching. I'm using OpenRouter with Novita AI as the preferred provider.
I’m using zen because I have a Claude subscription and just like dabbling with the other models and I was shocked at how little flash cost but it was noticeably not at the level I’d like my model to be.
For me MiniMax 3 has really hit the sweet spot of being very cheap, though more than flash, but I’d also very capable.
Opencode's free models have been fine for me, they're what I tried after Gemma 4 8B proved hard to persuade into usefulness (I want to revisit with 12B and messing with harnesses, but I'm happy for now).
I've been thinking a lot about this and my personal take right now is that at some near-medium future the models abvailable to run at home and the hardware needed to use them will be enough.
My baseline is sonnet 4.6. I think it's good enough for most tasks sincerly. So, from what I see, we are already at a point where we don't need frontier models for serious coding and debuging. Give it a couple of years and that level will fit 120B models.
At the same time, we saw the rise of direct acess memory systems like DGX or Stryx Halo that will allow to run models of this size for "cheap" in the medium term.
That's what I'm betting in. That in 2 years I can buy a system for about $2500 that will run a model that's similar to Sonnet 4.6 locally.
I might be spectacularly wrong though. But I'm willing to wait and use subscriptions/API calls for now.
What kind of usage chews through Claude Max x20? I use several agents with max effort in parallel and usually end up with something like 50% weekly usage. Fable almost allowed me to get to 70% but then they started resetting the limits mid-week and of course now ended the whole thing.
Hardware and provider juggling is a way to go, although I think it is also worth mentioning that the cost is not only the price-per-token, but first of all, the amount of tokens used.
Depending on what one builds, comprehensive documentation and applicable skills and memory tools often allow for a substantial reduction of tokens previously used by the agent to comprehend and remember what is being built
There’s a lot of Xeon chips for $10 on eBay. Too bad there’s no drive for cpu based inference. The data center will need to swap out the older gpu clusters so what does that do for hardware pricing on data center gpus? H100 are cheap enough but the power requirements make it a long term net negative for how much pay for power in California.
This month I've spent only 15 cents using DeepSeek API and my own coding agent. Three apps delivered to clients and currently working on a tournament management app for pickleball, padel and beach tennis. I love DeepSeek.
Yeah, although that is pushing every rate limit and no one knows what happens if you do that consistently? I think $4,000/mo is probably a good estimate for an individual dev doing synchronous coding agent work.
Yeah, I agree. I've been consistently getting about $1,000/month of value out of the $100/month subscription for OpenAI, and about the same for Anthropic.
Sorry to be that guy. I think the more precise wording would be that you get tokens which would cost $1,000/month at API pricing. Maybe (depending on the profit margin of the API pricing) you incur costs somewhere close to $1,000/month. And maybe your usage is subsidized by 900$/month. The value you get out of it is a whole other question. One that according to recent news, CFOs find hard to esitimate.
Maybe today but it's not a law of nature. It seems inevitable that AI models and coding agents will be fully commoditized eventually, just like computers, game engines, compilers, web servers, and so many other technologies have been.
At the end of the day, AI models are relatively small files that we run little CUDA programs on.
Fixed-price monthly plans ought to be sufficient for most people who actually review their spec and code, for building production-grade software that stand the test of time. A careful spec+review+iteration takes time, resetting the usage quota. Granted, security audits uses tokens too.
If you still need more tokens, odds that you're vibecoding unmaintainable throwaway trash.
With access to view usage for my org and conversations with developers, I think much of the high token usage is a result of people not knowing how to right size the model for the given task. The trend seems to be to pick the most powerful model and use it for everything. Based upon git metrics, I'm one of the top performing engineers at my org and I've yet to run into any overage or throttling on the $200/mo anthropic sub.
Is spending (metered money) even worth it? Perhaps for most I mean "beyond like a 30 bucks a month," but for me I'm literally not spending more money beyond my very cheapo 16gb video card.
No clue what y'all are doing, perhaps because I'm hobbying, and also I'm old and can perhaps do more of this by hand.
But I'm basically just doing what I did before, plus ollama self hosted and sometimes gemini and I feel like I'm going lightspeed beyond what I've ever done.
And I suppose this is still very fine-grained. I have it make a draft, then just have them fix/change it step by step?
I tried one of the bigger boys that can one-shot apps, which I guess is cool, but I'm finding it's just as hard to modify as if I just grabbed someone elses repo on github.
You can have opencode and switch between multiple providers based on the tasks you are doing on the fly, normal tasks use deepseek for example, hard one use gpt5 or opus4, and track the usage with something like codexbar or similar. Openrouter seems to charge extra on top of the api costs, same with zen ide, so keep that in mind.
> The first is to self host. You buy the machine, run open source models locally, and pay nothing per token after that.
In the good ol' days, we bought machines not only to run stuff, but to experiment.
I understand today experiments are limited. Inference is reasonable, fine-tuning is either niche or a stretch, and base training is impossible.
*That is bound to change*, and when it does, there will be an avalanche of hobbysts and amateurs poking at base training. They'll find optimizations no one found before, synthetize data no one ever imagined to synthetize, and when that happens we'll start getting libre models.
So, yeah. Right now, buying the machine doesn't pay off that well, unless you want to pioneer this stuff in severe adverse conditions (hardware prices inflated, etc). Eventually, it will.
I feel like I must have plateued and don't know what to do next to level up. I'm currently on the $100/month codex plan and it seems fine using 5.5-xhigh all the time. I think of what to do next, have a chat session to determine exactly what to ask for up to the point of being ready to implement, and then codex churns on a commit-sized task whereupon I briefly check it on my local dev server. If necessary I ask for a change. Then I ask it to commit and recommend the next step based off the spec. Oftentimes I have to "approve" an out-of-sandbox request anyway.
I haven't found anything that requires running all night. I could tell it to one-shot a big plan but given how often I realize I want an intermediary thing to be slightly different it seems like a waste of effort.
I'm guessing the next thing I should probably look into is some sort of machine vm I can tunnel my codex-gui requests to so I don't have to deal with the sandbox approvals (I don't want to give it "dangerous" access to my entire mac).
I don't understand what people are doing with their side projects that is leading them to churn through tokens so quickly, to the point of requiring two $200/month subscriptions and a bunch of token charges besides.
I’ve watched a bunch of layman videos where they create stuff with AI, these people burning through 12 hour tasks are literally not reading the output or understanding what it’s doing. Like they’ll ask for a program, and then right after it’s been created they ask the AI how to run it. Then when there’s a bug, they ask the AI what went wrong, or scrap the entire thing and switch model/harness and try again.
Here’s an example https://m.youtube.com/watch?v=xc1296HY8Fw&ra=m
It’s completely different to a professional workflow (what you described). It’s a toy for consumers
Amazingly, there are people out there (apart from creators), that work that way in their day-to-day job. I had the pleasure to work with such a person. After several months, he got removed from the position. He left a mess that hasn't been cleaned up completely to this point.
It won’t be long till employers get wise to this stuff, they just need to burned a couple of times.
It seems AI is good, great even at many things. But it doesn’t seem like it’s going to change the world as much as some people believe it will. And if it does it’s going to take time
Yeesh that sounds painful. There's definitely a fine line between vibe coding as a professional engineer and vibe coding as an outsider.
Well, if you believe the people who sell the tokens, you should be creating loops that keep yanking the bandit’s arm.
I have downgraded my Claude to the $20 one, and basically only use it for the web chat right now. For coding, I use DeepSeek @API Rates configured in Claude Code. I have spent around $4.8 for 320,000,000 tokens. I always felt like i was not using Claude plan, that i had to have the LLM working on something all the time to justify the price. Now with DeepSeek i don't think about it anymore. I don't feel bad when not using the subscription anymore, and i don't worry about limits as i just pay more. Where i really felt this was on running things in parallel as there are no hourly limits anymore!
I’m using $200 a month Codex working on a game for my kids for fun and curiosity since I’m a dev, I’ve played games, but I’ve never done dev for games. and have all night tasks but mostly they’re “spend time tending to and adding stuff to my 3D asset pipeline”. My RTX 5090 runs Trellis2 -> ultrashapes -> Trellis2 -> wiring up rigging and setting up animations.
But like 99% of that task is just Codex waiting for the output. So it’ll run for 12 hours but mostly it’s just setting lots of sleeps. I haven’t gotten close to running out of tokens. The $100 a month codex I hit usage limitations almost immediately, about 3 days in of working like crazy with 10 agents going at once, mostly coding an asset pipeline, I ran into my weekly limit and upgraded. So with the $200 a month plan at 4x more credits I haven’t hit any walls at all and can absolutely cook.
I have been on $100/mo claude and it has been churning out quite good software for months now. like i estimate what would have taken me three ish years, assuming i didn't burn out from failure (i would have). i only hit limits when i double fisted claude with my main project and my side project. just the other day i noticed i had been stuck on 4.5 because i failed to update the npm package.
Patches to existing sizable codebases and reverse engineering binaries both can run a long time and use a lot of tokens without wandering off into the weeds.
I'm on $100 Claude. I have a setup with bespoke local services that mitigates some high token consumption scenarios with local LAN services. I screen mcp's and hooks for cache poisoning. I run 100% on Opus with max effort, and never came close to hitting 5 hour or weekly limits before the Fable release. I am in Claude Code at least 20hrs a week.
I see people just completely wasting tokens with ridiculous setups, 100% hitting cache misses as well as dumping huge files into context all the time.
Just learn how these things work, or pay the price I guess.
> I don't want to give it "dangerous" access to my entire mac
I'm running Claude/Codex inside native macOS sandbox, configured with a simple script - https://github.com/sheremetyev/sandfence
always in "bypass permissions" mode - it works until task is solved, sometime 1 hour or more (which includes running tests etc)
recommend converting to https://github.com/apple/container
Linux VM doesn't run native macOS toolchain and requires copying files back and forth
Same boat here. I’m able to get a lot done on CC at $100/mo and feel like I’m not being creative or productive enough somehow when I hear of people blowing past that in a day.
I can get away with the $20/month plans, even on extra high, when I'm the one primarily calling the shots and writing code. I use the LLM for intelligent codebase searches, reviewing my code, refactoring that I direct, and scaffolding simple features for me to pick up.
I have also tried having the LLM write everything for non-critical, temporary projects that I thought were simple. It's amazing to watch how much the code explodes into spaghetti if you're not intervening. Every day the codebase would explode with more stuff: Extra abstractions, more arbitrary tests, harnesses to support those tests, leftover cruft that the AI stopped noticing and left in the codebase.
As it all accumulates, each new change request to the LLM turns into a journey through the mess it has created for itself. Even changes that I thought would be small could turn into 30 minutes of churning different tests that were added in the past and intersected with the change in an unexpected way.
So you can try to contain it by running more agents and spending more tokens to try to do clean-up passes and architecture rework, but it's a constant battle. You also have to be in the loop frequently to stop them when they're on a mission to do something you didn't intend, don't want, or recognize as a bad idea. If you're trying to get it done in the background, then you really just let it go and hope you can spot the problems later and have the LLM fix them.
None of these problems show up when you're doing these amazing YouTube or Twitter demo app posts where you go from 0 to something that looks like an MVP in a short clip. The LLMs are amazing at doing that. It's all of the problems that arise with maintaining it afterward that will start consuming tokens at rates you didn't think were possible.
I think this is also why we don't see many true vibe-coded production apps: It's really easy to get something started with an LLM and even shipped to production, but the technical debt load is so staggering after a while that your "agents" are spending hours trying to unwrap everything to do every single feature request.
> The first is to self host. You buy the machine, run open source models locally, and pay nothing per token after that.
Power is not free.
What I’ve found is that you’re basically paying a premium for privacy, and that’s worth it for me.
Luckily I needed a new laptop and I bought an M1 Max secondhand from a friend quite cheaply because it was fast enough to recompile something else I am interested in.
So for me, there is no additional hardware cost; it was acquired in replacement.
I run the AI models at home on this kit because I want to; I'll use openrouter if I need to.
I accept the economics of this article are right. But I feel so incredibly sad about this outcome that we're now just to be people caretaking machines that do the job we loved that actually I am not sure that exercising this nuance is going to matter in the long term.
It turns out it is a mistake I have made in my life — now really unfixable because I am a bit too old — to believe that I will always find enough fulfilment in my work to offset the absence of personal fulfilment elsewhere; I have always enjoyed being able to help people directly by doing a thing I love and I am good at, and that has kept away the sadness of finding it difficult to build a conventional family life to enjoy.
I assumed I would always find some new way to find that enjoyment, but even the slim enjoyment from being able to explore this stuff on my own kit in my own terms will not be enough if the pendulum does not swing back towards human effort.
It is a dismal world we have made for ourselves. Lately I have found myself dreading growing too much older in it.
You sound awesome. Just venting? (b/c curious if friends can fill your heart abundantly, & we know we're never too old to make new friends!)
> dreading
Even avoiding political headlines (OK, at least articles), plenty of cause for dread, so I keep re-focusing to avoid despair. Easier said than done innit!
Can't kill my hope for the future though. One day, all the good stuff shall prevail (morality, intelligence, love & kindness)... maybe not permanently, but a Star Trek future is there somewhere (& they had their troubles but it wouldn't be a dreadful situation overall). Sharing with you in case it's even slightly contagious!
Also, I would anticipate at least a 5 year lifespan for a current generation card. The 3090 is still respectable simply because it has 24GB of RAM which, for years, has been the limiting factor for ML at home. If you got a 6000, sure it’s going to cost 7-8k, but the resale value is likely to be very good. Even the 3090 is 50%+ of RRP still. And if you’re not doing LLMs, it’s an interesting value proposition for “classic” CNN vision model training. You can fit enormous batch sizes on 96 GB. The biggest reason to upgrade is perf/watt has about doubled (eg 4000 pro Blackwell is half the 3090 for similar).
People tend to assume the capex is thrown away but as we’ve seen with RAM, don’t be so sure you won’t be able flip it if you need to.
Actually if you have solar, it kind of is.. so prIvAt AI compute gets defacto cheaper during the day?
If you have solar, it is not, because you have battery and equipment degradation from cycle charging, c’mon man…
I would agree with you if you said it was vastly cheaper overall (with the initial equipment investment amortized over time) compared to The Power Company.
In many states, even if you are generating electricity and selling it back to the power company, they still gonna charge you normal rates of usage because greed.
If you go off grid, you have bigger things to worry about than how to power your AI cluster. It’s manageable enough if you have land but that’s in scarce supply.
> if you have solar, it is not, because you have battery and equipment degradation from cycle charging, c’mon man…
no, the rate of that is pretty independent of use. unless you live in a place where selling energy back rules are designed to screw the solar owner (California)
California, Arizona, Texas, most of the southern states…
>> Power is not free.
There's actually an interesting thought experiment here: if it takes you a full day to build something that AI would otherwise build in a day, do you end up using more power, or less? What is the break-even point, purely from a power consumption perspective?
If an identical task takes a day on both sides, then the human route uses less energy, surely.
Brains are thousands or maybe even millions of times more fuel-efficient than computers and you are alive for the whole day either way, right? You probably eat about the same even.
The reason executives think AI is more efficient is that it more space efficient than a human and doesn't demand to be paid or work only a set number of hours. Everything with computing is more efficient if you resent having to give money to other humans. If they could just not have you be alive when they don't need you, it'd possibly be different.
Even though I think at a typical British freelance rate and a truly unsubsidised token price, the AI is possibly more expensive than me. And as a freelancer, from their perspective I really am not alive until they need me. (This is what it often feels like)
The reality is the human and the AI aren't used to build the same things anyway so it's a comparison you can't really make.
Brains are efficient, but civilized humans aren't. In the USA, adults consume at a rate of about 10kW -- only 1-2% of that being the human's metabolism, the rest being HVAC, electrical devices, etc.
For comparison, a modern frontier model like Gemini 3.5 Pro consumes about 15kW -- so only about 1.5x the fully loaded human. In an 8h workday, that model would crank through ~80M tokens (~$5k at API prices). That's ~4 major refactors of a 10k LOC codebase, so probably not a very realistic comparison to a single human dev.
I think a more useful comparison, based on my experience, is that an engineer with AI support can get one 8h day's worth of unassisted work done in 1h. So, the 25 kWh consumed during collaboration (conservatively assuming I keep the GPU hot for the whole hour) frees up the remaining 70 kWh I'll draw down for the day to be spent in some other way.
You forgot to mention that it takes a lot more energy to train that human before they're able to work.
Studies on grandmaster chess players indicate that at most you burn 10% more calories when engaged in deep thought than when you're at rest. So the energy "attributable" to an hour of knowledge work is like 10 calories (average sedentary calorie burn is like 80-100 per hour; add a max of 10% for the thinking gets you 8-10 calories). A pound of potatoes is like a buck and is about 320 calories. So you're looking at like 3 cents an hour at most to cover that energy burn. It's definitely even less; I certainly don't think as hard as a grandmaster chess player.
Then, assume power costs 20 cents per kilowatt hour (US avwrage) To match the human 3 cents per hour, you need an average of 150 watts of power drawn per hour. That's in the range of a budget graphics card, but not much past there.
However, if you sleep instead of sitting around, you can probably make AI cost competitive. Sleeping drops your metabolic rate by more, and lying down in bed (as opposed to sitting) also reduces calorie burn. Combined, you can reduce your burn by like 30 calories an hour. At the new 9 cents per hour human cost, you can afford to run a higher end graphics card at ~450 watts per hour. That puts you in RTX 3090 range.
What would you do for the rest of the day, power off your devices and go for a long bike ride?
Speaking personally: yes. That's literally what I'm planning to do this afternoon because it's noon and I'm already done with the coding tasks I had on my plate today.
Luckily the future is absolutely going to be that star trek one where technological abundance means we are all wealthy and have free time to develop personally, and not the future where all the money bubbles up into the hands of a thin-skinned malignant narcissist who wants to play with launching rockets and provoking racial violence /s
I'm assuming that you need to feed the human being (i.e. you) regardless of whether you use that human being for writing code or not. So, by this metric, there is simply no breaking even point. The cost of human + AI is always going to be higher than the cost of human.
I'm in Florida and am already using AC, so if not "free", definitely "negligible."
work at a cafe.
> Power is not free.
its ~free if you have home solar.
Solar panels are free and never break and have an unlimited life expectancy?
Solar panel breakage doesn't depend on a graphics card.
What are people doing at home? I have like 5 different apps I code on the $20/month Claude plan and like sure I can hit rate limits but - What are people doing to burn through $3k in tokens?
Same for me. $20/mo is just fine and I use it to code daily.
I suspect the people that burn through tokens have several subagents and 50 skills loaded and 40 MCP tools. All those load up the context on every single turn.
YMMV but automations eat through the $100-$200 plans, which burn thousands in tokens alone.
I have hourly automations for root cause analysis on customer support issues, daily automations for eg log analysis, weekly & monthly automations for SEO/miscellaneous KPI tracking & actioning.
I will say, when I was building side projects that were 1) fairly well defined in scope and 2) without users/need for automations it was much easier to stay under $20/mo plan limits. Now I regularly hit weekly limits and need multiple Max plans
I invested about $4,000 in an NVIDIA DGX Spark several months ago. 128 GB of unified RAM, and the NVIDIA GB10 chip. With the RAM, the several CPU cores, and the 4 TB NVMe SSD, it's a very capable ARM64 Linux computer even without the GPU, and so far I've mostly been using it as such. But I wonder, what's the most capable model, specifically for coding, that can run well on that hardware?
https://www.canirun.ai/?status=tight might answer that question
That site doesn't seem to consider the quants. So useless.
Deepseek v4 flash is shockingly strong for its size and reportedly runs well on that hardware.
Better than qwen3-coder-next? That's the one that has given me the best results so far.
>Around $400 a month of plans buys roughly $2800 of API usage at list prices, which is a real bargain right up until you hit the ceiling. The plans are metered, and any large AI native workflow will chew through the included tokens fast
I don't think that's true at all. I'm doing 8-12 PRs a week at work, all primarily Claude Code, and the usage at API billing has never broken $500/mo.
Maybe yours is not a large AI native workflow?
I find just going via Deepseek's platform API directly, using their V4 flash model, and hooking into a harness like Opencode more than acceptable. Think I've spent maybe $10 over a couple of weeks.
I did explore self-hosting models but hardware right now is just too expensive.
Directly at DeepSeek? It was my understanding (but I didn't check) that some other AI operators were providing (some of?) DeepSeek's model for cheaper prices.
Still, that's interesting. What do you get for that price? Only coding, or also e.g. image generation?
For me, investing in hardware seems to be the way to go.
I learned coding nearly 24 years ago and still learning new stuff all the time. At no point in time I had to rely on a subscription model to learn and do new stuff.
If LLM and agents are the default tools for coding and building software, at least for next few years, it seems like a no-brainer to invest $2000-3000 on hardware, like a Halo Strix PC.
I wondered if there might be a no brainer "free" option on discarded hardware.
I have a GTX1080ti which i think is circa 2018, it's unused, more than paid for itself over the years, owes me nothing at this point so the hardware is free.
It runs Gemma e4b multimodal, qwen 3.5 8b or the qwen 4b embeddings models well enough (40+ t/s for the LLMs).
The machine consumes 350 watts at the wall when under load (3 watts when sleeping, 80w at idle). Electricity costs me £0.035GBP/kwh which is cheap for the UK (load shifting via house battery).
144k output tokens for around 1pence (and takes an hour to do that in theory).
It's only JUST cheaper to use than the far more capable deepseek v4 flash model despite the free hardware and ~10x cheaper than normal electricity.
Yes and no. Hardware does lock you in. Granted, I am happy with my 128gb of shared memory, but I am mildly concerned that it actually is more expensive now than when I bought mine. It does not bode well for the future; not when combined with recent WH admin moves on Anthropic and the reality that next batch of good models may require more than 128gb to run well.
edit: I am not dismissing local. I am one such user ( though I have subs too ), but one has to be clear eyed about the trade-offs.
$3k isn't getting you frontier model capability. It's barely getting you any capability if that's split into buying an entire PC rather than just GPUs.
With you here. I'm using my cheapo 16gig vram card I picked up a year or so ago, and I'm like -- yes, I percieve that you can pay for way more tokens per second that I can do at home.
But that feels like measuring productivity in lines of code. For what I'm doing, I'm not seeing the benefit in any subscription.
Sure, I can't one-prompt a whole new boring CRUD app, but oh well.
3k? Try 10
> The upfront cost is steep and the models you can actually run at home are weaker than what the frontier labs ship, so this only pays off if you can keep the rig busy with long running tasks where a slower, cheaper model grinds away overnight. Most people can’t keep a home machine that loaded, and the hardware you buy today may look like a bad bet in a year.
Oh, so this is not a post about AI coding at home. It's about vibe coding at home.
There's a lot I disagree with in this post, but I'm posting this from a home computer with 64 GB of RAM and no GPU. I do lots of AI coding while spending very little money. I run Gemma 4 26b (mixture of experts) and Qwen 3 coder with Ollama. I use Github Copilot code completions. I use the Gemini and Mistral API free tiers. I have a Gemini paid API account. It's now prepaid, so you don't have to worry about an accidental $1000 bill. You can do a lot of things with Gemini Flash Lite 3.1.
None of this is burning through tokens to create an expensive blob of spaghetti code, but it does qualify as AI coding.
My sentiments too. I'm using Qwen 3.6 35B A3B on a machine with 64Gb ram and a 24GB 5090 (an Alienware 16 Area51 I bought, serendipitously, about 15 seconds before the idiots preordered all computers for the next 3 years and ruined everything).
You can't "slop cannon" vibe code with it, but this is personal code I want to not be spaghetti, so I'm not trying to vibe code. I just want to get instant retrieval of all stack overflow and reddit posts in a chat box, and for it to be able to spare me the physical pain of actually having to type out typescript code (I am a BE dev with negative patience for all frontend) and fuck around endlessly debugging obscure docker problems (I like docker, but, no patience for it having annoying problems and endless quirks). And this model does that really well.
Can I run something comparable to Opus 4.6 locally yet? I keep hearing conflicting things. If I can spend 10k to do that I would cancel my subscription. The problem is I don’t wanna spend the money to find out myself.
If you want frontier-level, the economically reasonable option is OpenRouter or a direct sub to frontier-of-your-choice.
The reality is that they do not offer configurations that would allow a consumer to run that much VRAM on a single setup to protect datacenter margins. Apple used to, and they stopped, those devices are going for ~$20k+ each on ebay now.
You can get very, very capable models on a 3090/4090/5090/6000 series card. But if you want 'frontier level' you are investing ~22k at a bare minimum if you go new. Used you can probably build your own server for much cheaper up-front cost but it's likely going to be 4-6x+ electricity usage.
There are also significant economies of scale (namely: utilization and batching), which tend to make inference on a shared server more economical even after the operator takes a cut.
You can use batching on consumer hardware, it just requires a KV-cache efficient model (or short context only) and keeping multiple inference flows running in parallel. This is most useful in combination with streamed inference, since the compute intensity of decode with those newer KV-compressed models is high enough that you have limited compute headroom when running at the speed of RAM.
I truly think by 2028 we'll have integrated chip systems that'll be able to run opus 4.8 level models at ~500 watts at acceptable performance. Honestly I think now is the worst time to invest in AI hardware. Get your harness ready and processes perfected with hosted models, and wait a few years to buy hardware to transition to running models locally
Burning weights onto a chip in an efficient way and exposing that via USB would be acceptable for a good enough model tbh
This is pretty close to what Taalas is doing.
Trying Taalas is almost scary, there is something unsettling with that speed! Even with that small model, because of the speed, you could run hundreds of sample runs in a second, and pick from the best.
Can't wait for their next release!
if such hardware becomes available, it will be bought by the data-centers, just like they buy all the RAM today
Honestly I think now is the worst time to invest in AI hardware.
That position is not without its own risks, though. Maybe Opus 4.8 will run on a single chip by 2028... and maybe you won't be allowed to touch it.
And what if Xi makes a play for Taiwan? That would be stupid, but so was invading Ukraine with tanks from Temu, and it still happened.
> so was invading Ukraine
the difference is that Putin's hand was forced by age, (possibly) illness, and the last several decades of how he chose to run his country. Putin's power base is a relatively small group of elites and oligarchs who would happily snuff out the man who pushes them out of windows if they get too uppity, if they were given the chance. He needed the cover of war to maintain the fiction of his type of strongman "only I can save us" leadership.
Xi's power base is the simple fact that his leadership has transformed China into the #2, and now because of Trump possibly soon the #1 world superpower. He has also acted aggressively in the last decade to find and remove corruption and prevent individuals from accumulating the kind of wealth and influence that could threaten his power from outside official Party channels. Of course, as I'm not Chinese myself, I have no clue what the internals of Party politics actually look like. But as an outside observer it seems clear that Xi et. al. do not actually need Taiwan for anything other than national pride. They know the US would go to the mat to protect it as TSMC is extremely vital to US military power. And since China cannot compete in that arena and has too much to lose, they instead have focused on weakening the US from within, quite successfully of late.
By the time China finally takes Taiwan it will be with little fanfare and little consequence - they won't touch it until the US either has lost its military capabilities, or the US has its own internal chip industry. Anything else is an existential risk for the coastal cities that are China's entire economic advantage.
10k will not get you anywhere near opus or sonnet. It's simply not possible for mere mortals currently.
> Can I run something comparable to Opus 4.6 locally yet?
Sadly, no. The best comparable thing you can get is about Sonnet 3.7
i spent 8k and get close to a 2-3x slower sonnet. running 2x spark deep seek v4 flash
Some benchmarks have shown Kimi K2.6 within error-bar distance of Opus 4.6, and you can run it on eight RTX6000s. Right now it's not possible to set up a machine like that from scratch for less than $100K... but right now it's also hard to put a price on autonomy.
You need a lot less than that if you're willing to stream the model from SSD. At that point, the best machine is probably a cheap old-gen HEDT with lots of PCIe lanes to attach cheap NVMe storage to, so as to stream the model at reasonable speed. That's expensive but not $100k expensive!
Best you could do is connect two Mac Studio M3 Ultra 512G RAM each with Thunderbolt. Then theoretically you can run frontier Chinese models (but not Deepseek v4 Pro yet). That would be about $20k.
But - good luck finding them. Apple discontinued the model a few months ago. And more recently, even 256G model was discontinued. Big AI really really does not want people to get off their needle.
DeepSeek V4 Pro is ~800GB total at native quantization (1.6T params with most being 4-bit) so it can run on the hardware you mentioned. There is also a 2-bit version that will run on a single 512GB machine. SSD streaming also makes lower-end hardware viable to at least test the model, if not quite run it usefully.
> Do that well and you can build what a team of twenty engineers would put out in a month for around a thousand dollars.
What does this look like after 6-12 months? Like, how much code are you trying to write total?
Maybe it just doesn’t click in my mind, but sometimes I wonder about how much work people are trying to do and how they actually have enough to get done so quickly in such a short amount of time.
They prefer to work harder and not smarter. Forever hill climbing to nowhere.
I've never worked on a complicated codebase that started out that way until the rest of the business concerns and office politics came into effect. People may not like it, but the bureaucracy is far and away more valuable than the core functionality.
Mature codebases are years of people thinking of all the possible gotchas while solving their acute pain points. This is not fluff, but the living and breathing part of it. Without that code, it's just a machine barely doing stuff in the most obtuse ways possible that nobody wants to pay for.
I would argue that they're putting LLMs to work on that finer detail stuff, but AI is still far too dumb. No, what they're doing is playing with their skinner box.
AI coding at home literally costs $100/month. I'm wondering where $400 is coming from? $100 is more than enough for "coding at home", IMO. I rarely face the limits, and when I do it's just a time for a quick walk anyway.
Man I’m using the $20/month sub and it works just fine for me. Granted, I have a family and house and lots of obligations so by the time I hit the limits some other task is due before I can return to coding. If I hit the limits before I have something else to do then I just code by hand or review what has been generated until I can use the agent again. Reviewing agent code is a good way to learn too, agents have shown me different approaches than what I would have done and they’re definitely worth thinking about. Also, fixing their mistakes has helped me write better prompting although being a team lead for half a decade has taught me how to specify what I want very clearly and cc gets it right most of the time haha
About interruptions, one thing AI assisted coding really helps with is coding with constant interruption. I can leave CC for half an hour and return then tell it I had to step away, catch me up, and proceed. This works well for me.
I think this is only going to become more relevant. I'm personally a $200/mo Claude Maxer and I know that the usage I'm getting on Opus 4.8 Max and (until they yoked it out from under me) Fable 5 is way, way more than what I'm paying them. At some point, this will turn usage-based and I will be hammered on it and probably forced to look at self-hosting. I think while the caps are there, even at $200, it's honestly not too bad if you're coding value into the market, but as soon as those caps come off for retail AI users, we're all going to have some tough choices to make.
If your job becomes writing complex specs to make an LLM write code, you've not optimised anything.
In fact all you've done is add a business cost.
I think someone could find some way to use the smaller local models to write code. Some kind of framework or harness or language or something. But not too many people are working on that because the big models are pretty cheap and a lot better.
Maybe one possible path(to make weaker models highly capable) is making the job of the llm as easy as possible.
I wonder if part of the solution is building/finding the right libraries, with the right documentation/language/API(one that plays well with LLM's) and maybe creating some synthetic data around them - to make it very easy for the llm.
And maybe there could be a business model around creating those libraries.
So in my limited experience: The smaller the model, the bigger the harness. The biggest issue becomes the context window. For big models you can kind of just give it bash access and let it run... while with the smaller ones you need to fully manage the context in each LLM call.
If you can ask the model for a specific function; with a spec design (typed languages help too) then the small models are great! I have had good progress with generating small python modules for example, but you need verification rounds to catch issues.
So test driven design + a good spec sheet + a very detailed todo.md (or even better if its todo.json because then the LLM does not need to manage it, you do from the harness) is your best bet for small models.
I think as well there might be "algorithms" that can work with local LLMs. With local LLMs there is a small context window, but not that much cost per token. So perhaps there is a way to do lots of small prompts that work in a sequence to produce a result.
Like perhaps you could produce 5 versions of a piece of code, and then compare them to choose the best.
Also if the local LLMs can call tools, maybe you can use static analysis tools to catch errors and try again in a loop or process of some sort.
There also might be certain languages that work better because those languages have better static checks.
Yes. LITERALLY THIS. I do this! Not hypothetical.
I'll write a detailed prompt for a function, hand it off to 5 or so models (all of which are on my local machine), wait about 5 min and then compare.
I mean, this is what I'm doing. I'm guessing my process is very different because I'm holding the hand of the project way more along the way, but even that to me probably makes for a more enjoyable.
Which is to say, I might use AI to do an outline/organizational , but I'm prompting every chunk of code "one-by-one," (e.g. at about the "function" level) which still feels lightyears ahead of what I used to do.
I recently made an AI Agent and surprisingly coding with DeepSeek V4 Flash is quite cheap. It probably has to do with the aggressive prompt caching. I'm using OpenRouter with Novita AI as the preferred provider.
Deepseek v4 via deepseek themselves is significantly cheaper.
Because (1) Huawei collab and (2) vLLM etc dont implement half of the inference optimisations deepseek proposed in their paper.
Same here, deepseek v4 flash on opencode go. It's cheap, fats and good enough to follow my instructions
I’m using zen because I have a Claude subscription and just like dabbling with the other models and I was shocked at how little flash cost but it was noticeably not at the level I’d like my model to be.
For me MiniMax 3 has really hit the sweet spot of being very cheap, though more than flash, but I’d also very capable.
Opencode's free models have been fine for me, they're what I tried after Gemma 4 8B proved hard to persuade into usefulness (I want to revisit with 12B and messing with harnesses, but I'm happy for now).
I've been thinking a lot about this and my personal take right now is that at some near-medium future the models abvailable to run at home and the hardware needed to use them will be enough.
My baseline is sonnet 4.6. I think it's good enough for most tasks sincerly. So, from what I see, we are already at a point where we don't need frontier models for serious coding and debuging. Give it a couple of years and that level will fit 120B models.
At the same time, we saw the rise of direct acess memory systems like DGX or Stryx Halo that will allow to run models of this size for "cheap" in the medium term.
That's what I'm betting in. That in 2 years I can buy a system for about $2500 that will run a model that's similar to Sonnet 4.6 locally.
I might be spectacularly wrong though. But I'm willing to wait and use subscriptions/API calls for now.
What kind of usage chews through Claude Max x20? I use several agents with max effort in parallel and usually end up with something like 50% weekly usage. Fable almost allowed me to get to 70% but then they started resetting the limits mid-week and of course now ended the whole thing.
> and the hardware you buy today may look like a bad bet in a year.
3090s and 7900s are going well so far.
Next year an Arc Pro B70 won't produce you less tokens than today.
They aren't fast but if you have flows where you can make money with them - they are a bargain in terms of price per Gb.
Ha just wrote a post[1] about a sort of 4th option - max out cheap compute to create more tangible things that can be used/run locally.
1: https://news.ycombinator.com/item?id=48519181
Hardware and provider juggling is a way to go, although I think it is also worth mentioning that the cost is not only the price-per-token, but first of all, the amount of tokens used.
Depending on what one builds, comprehensive documentation and applicable skills and memory tools often allow for a substantial reduction of tokens previously used by the agent to comprehend and remember what is being built
There’s a lot of Xeon chips for $10 on eBay. Too bad there’s no drive for cpu based inference. The data center will need to swap out the older gpu clusters so what does that do for hardware pricing on data center gpus? H100 are cheap enough but the power requirements make it a long term net negative for how much pay for power in California.
This month I've spent only 15 cents using DeepSeek API and my own coding agent. Three apps delivered to clients and currently working on a tournament management app for pickleball, padel and beach tennis. I love DeepSeek.
Did you just copy-and-paste an AI response an post it on your blog?
I use copy & paste with a pro subscription. I guess I'm a bit behind in terms of tool use but it works great for me.
Similar story. I did have a pro subscription as a trial. I'm finding the free tier is as good(for my purposes) as the paid model.
"Around $400 a month of plans buys roughly $2800 of API usage at list prices, which is a real bargain right up until you hit the ceiling."
I realize this text is just slop but it never stops being a "real bargain" at any point.
And it's more like $200/mo for $4000+/mo in tokens. You can also buy additional subscriptions.
There's no sense in running local models or doing anything else as long as VCs (and soon the public markets) are willing to pay your bill.
SemiAnalysis pushed this to the limit and managed to get $8,000 of tokens from a $200/month Anthropic plan and $14,000 of tokens from a $200/month OpenAI plan: https://twitter.com/SemiAnalysis_/status/2064815044085318040
Yeah, although that is pushing every rate limit and no one knows what happens if you do that consistently? I think $4,000/mo is probably a good estimate for an individual dev doing synchronous coding agent work.
Yeah, I agree. I've been consistently getting about $1,000/month of value out of the $100/month subscription for OpenAI, and about the same for Anthropic.
Sorry to be that guy. I think the more precise wording would be that you get tokens which would cost $1,000/month at API pricing. Maybe (depending on the profit margin of the API pricing) you incur costs somewhere close to $1,000/month. And maybe your usage is subsidized by 900$/month. The value you get out of it is a whole other question. One that according to recent news, CFOs find hard to esitimate.
Even if they were making a profit, their scale and expertise will obviously give you a cheaper product than what you can build.
Maybe today but it's not a law of nature. It seems inevitable that AI models and coding agents will be fully commoditized eventually, just like computers, game engines, compilers, web servers, and so many other technologies have been.
At the end of the day, AI models are relatively small files that we run little CUDA programs on.
Fixed-price monthly plans ought to be sufficient for most people who actually review their spec and code, for building production-grade software that stand the test of time. A careful spec+review+iteration takes time, resetting the usage quota. Granted, security audits uses tokens too.
If you still need more tokens, odds that you're vibecoding unmaintainable throwaway trash.
With access to view usage for my org and conversations with developers, I think much of the high token usage is a result of people not knowing how to right size the model for the given task. The trend seems to be to pick the most powerful model and use it for everything. Based upon git metrics, I'm one of the top performing engineers at my org and I've yet to run into any overage or throttling on the $200/mo anthropic sub.
I had no idea git metrics could show your best performers
It could put managers out of a job, without AI too, so they prefer to not use it.
> Do that well and you can build what a team of twenty engineers would put out in a month for around a thousand dollars.
As usual, an extraordinary claim without an extraordinary evidence: https://stephen.bochinski.dev/apps/
Is spending (metered money) even worth it? Perhaps for most I mean "beyond like a 30 bucks a month," but for me I'm literally not spending more money beyond my very cheapo 16gb video card.
No clue what y'all are doing, perhaps because I'm hobbying, and also I'm old and can perhaps do more of this by hand.
But I'm basically just doing what I did before, plus ollama self hosted and sometimes gemini and I feel like I'm going lightspeed beyond what I've ever done.
And I suppose this is still very fine-grained. I have it make a draft, then just have them fix/change it step by step?
I tried one of the bigger boys that can one-shot apps, which I guess is cool, but I'm finding it's just as hard to modify as if I just grabbed someone elses repo on github.
You can have opencode and switch between multiple providers based on the tasks you are doing on the fly, normal tasks use deepseek for example, hard one use gpt5 or opus4, and track the usage with something like codexbar or similar. Openrouter seems to charge extra on top of the api costs, same with zen ide, so keep that in mind.
> The first is to self host. You buy the machine, run open source models locally, and pay nothing per token after that.
In the good ol' days, we bought machines not only to run stuff, but to experiment.
I understand today experiments are limited. Inference is reasonable, fine-tuning is either niche or a stretch, and base training is impossible.
*That is bound to change*, and when it does, there will be an avalanche of hobbysts and amateurs poking at base training. They'll find optimizations no one found before, synthetize data no one ever imagined to synthetize, and when that happens we'll start getting libre models.
So, yeah. Right now, buying the machine doesn't pay off that well, unless you want to pioneer this stuff in severe adverse conditions (hardware prices inflated, etc). Eventually, it will.
Another update for codex users they let you accumulate resets which greatly adds to the mileage
I don't think its feasible to have something comparable to these frontier models when they are increasing usage and lowering token costs