I can only recommend to regularly measure how many tokens a harness+model combination uses for a certain task
There are huge token efficiency/bloat differences between agents while working on the same tasks, using the same model, in the same environment
Yesterday I ran 10 agentic tasks using GPT 5.6 Sol in an ubuntu 26.04 vm a couple of times with different harnesses and got vastly different token usage.
I'm not surprised that Claude Code is not optimized for an OpenAI model but I was still quite shocked re how much of a difference the harness makes.
Disclaimer: I'm working on 'smol' which is a minimalist harness but it's really nothing special, just a minimal system prompt, no skills files, only tool is shell
Do not underestimate how much popular harnesses are spamming the context window. The context window is very important.
Claude Code injects a ton of tools into the system prompt, including their “memory system” that’s like 10k+ tokens. Depending on your task shape, this can easily double your task cost (e.g. a low context-using job that takes many turns, like a monitoring loop).
You should use --disallowed-tools to prune any tools not needed for the task. Note that this is also a perpetual game of whack-a-mole since they’re always adding new tools.
> Claude Code injects a ton of tools into the system prompt
It's so unfortunate they don't let you use the subscription with other harnesses anymore - since even if I used OpenCode they'd still get a bunch of useful data from the API calls, meanwhile I could stretch their tier limits way further.
You still pay cache token costs on API calls. Cache cost/token are 90% lower, but you pay it every single turn.
I’m not sure if they let you skip the cache write cost on the first turn. That would imply cross-user caching infrastructure or special casing the default system prompt to give you a discount. Maybe? Away from the computer but you could try a “hello” in a fresh session and see what was billed.
What tools are you using to run the comparison? Or are you just running the prompts and checking them with something like ccusage?
I’m asking because I’ve been looking for agent harness comparison tools too. I’m interested in more than just the inputs and outputs—I also want the system prompts, traces, and tool calls. It’s useful to understand why Codex, for example, uses more tokens while Pi doesn’t.
Fewer tokens aren’t necessarily better if the agent skipped important checks. On the other hand, using more tokens could just mean it’s overthinking the process. Either way, seeing the full execution trace for the same task is really valuable.
I built an ad-hoc custom comparison framework to inspect system prompts, caching behaviour, tool call outputs, exact api requests and responses and so on
I agree fewer tokens is not necessarily better but a bit counter-intuitively often the harness using fewer tokens is not only done faster but has better results
(that said: of course check the results, look at the full traces, agree!)
Running some local models and wondering what insanity was consuming 100k+ tokens to respond to "hello world" was rather eye-opening, yeah. Full of pointless fluff.
It removes money from your pocket into theirs faster than Claude will take money out of your pocket and into theirs. I was a Windsurf then Cursor user and even provided product feedback to the Cursor team on various aspects of their product but the costs for using it killed the value.
1) It's still an IDE. Because of pricing I mostly use Codex, but I always have a VSCode/Cursor IDE open, thus have to juggle between the two. Working directly in the IDE is more comfortable. For full on vibecoding that might be worse, but when you want to do a deep review of the changes, an IDE is way better than reading a diff on github.
2) It supports every model. It's often very helpful to try different models when you don't like the result of the first.
The Claude plugin for VSC is amazing and basically "solves programming" for me. It's the primary way that I write code now; I even stopped using Claude Code.
Recently they also seem to have made big changes that makes Cursor worse for editing code by hand. I'm not sure what they are trying to do, but it doesn't feel like vscode fork anymore.
I'm starting to consider alternatives.
To answer your question though, to me your workflow seems cumbersome. Cursor is more integrated and more frictionless.
I don't think it offers any. I got tired of switching between Codex and Claude plus I wanted to start using glm and kimi so I switch to pi. Then I got tired of VSCode eating up battery and RAM so I moved to Neovim which has barely any efficiency downside anymore since most code is not written by hand and if you know how to use it well it was only maybe 10% less efficient than VSCode to begin with. I am not sure why a heavy desktop application would be better than nvim + pi in terminal which are both super extensible via vibecoding and very light on resources.
Huh, never thought about that 2nd point - I'm also transitioning to pi, but didn't think to change to neovim as well. I guess your logic holds for claude code users as well though.
Cursor Tab is nice when you're hand-editing stuff. Not sure it's worth $20 though. I'd totally buy and forget a $5 Cursor subscription just for Cursor Tab.
The $20 price point is just too competitive now and I'd rather use the Claude/Codex plugin over Cursor's agentic coding sidebar.
Does that mean you’re paying api prices + cursor markup (?), or can you somehow piggyback on your Claude subscription? Fable via the API is really too expensive
This is basically what I did. In 2023 I moved from VS Code to Cursor. And in 2025 when Opus 4.7 (or was it 4.6?) did that big leap in December, I switched to Claude Code and VSCode for any edits. I come from Sublime Text so those shortcuts are hardwired in me (which was also the defaults in VSCode). When Cursor hijacked literally every CMD key combo, I got fed up and switched back to VSCode. Maybe I should just go back to Sublime Text now lol, since I really just need a blazing fast code reader.
Saw this at work yesterday. Absolute insanity to do something as user hostile as removing the cost of the service. There's no way to attempt to brand this as anything but negative for the user, and positive for the company.
Gotta justify a $60B purchase of an IDE and (at the time) a single, decent model.
This month I've noticed that my Cursor model use has gone up faster with Composer 2.5 - I have bothered with Grok yet. Despite the advertised twice-the-usage on Cursor own models I'd say it's burning faster.
Cursor has been my corporate vendor for accessing non-anthropic models. Without cost insights and with no ability to proxy API requests its value drops dramatically. They fought hard for a renewal with legacy pricing mode then broke the deal soon thereafter. I’ll be extremely clear to leadership that Cursor usage should be minimized and not renewed. I frankly don’t trust them with IP either, and you shouldn’t either, especially if you’re demonstrably not in a big enterprise.
Cursor was a great introduction to agentic engineering but I've learned their Claude pricing is largely just batch purchases. Their real moat I think is Composer 2.5 because both their agentic and IDE experiences fall short of Codex and Claude Desktop in my opinion. I think their sales will tell you economically they make the most sense and I would probably agree, but cost isn't everything especially when the spread isn't that wide. At this moment, capability is really king.
These days I'm using Codex and Claude Desktop with Zen when I need to look at code. Codex's real time audio chat feature (not dictation) is also second to none when paired with their agentic flow.
Grok 4.5 is them as well now I guess since they're owned by spacex now. I'd consider grok 4.5 the sonnet and composer the haiku (both capable, fast models).
I’ve been working on a project to solve this problem with Claude code, codex, etc by recording usage and translating to $ in realtime. Looks like I’ll have to add Cursor support now.
I am shocked, shocked, that a company that is part of SpaceXAI would become hostile to its existing paying users and partners. This has never happened before in history.
cursor is a scam. they pissed me with their UI changes & shady pricing, so i tried claude code recently for the first time and instantly regreted i havent tried earlier
Cursor you have a beautiful comeback story, you have a wonderful model with Composer 2.5, and a terrific behemoth with Grok 4.5 now. A top-tier dev ux with the Cursor agent app, why on earth are you squandering this opportunity and behaving this way?
You came back from the dead pretty much and now you're pissing it away for what exactly?
Do not spite your individual developer customers or you will perish yet again.
The PM comment above matches what I've seen. Hiding a usage number doesn't remove the anxiety, it just removes the user's ability to verify it, so people pad their mental estimate upward and quietly disengage. For anything usage-billed, trust in the tool is capped by trust in the number, and you can't build that trust by making the number invisible.
The move that actually works is the opposite: make it legible. Show the breakdown, show what drove the spike, let people reconcile it against their own expectation. A scary number you can explain beats a hidden one they have to guess at.
Removing the CSV export is the part that stings most, because that's the one thing power users reach for the moment they stop trusting the dashboard.
I can only recommend to regularly measure how many tokens a harness+model combination uses for a certain task
There are huge token efficiency/bloat differences between agents while working on the same tasks, using the same model, in the same environment
Yesterday I ran 10 agentic tasks using GPT 5.6 Sol in an ubuntu 26.04 vm a couple of times with different harnesses and got vastly different token usage.
https://x.com/__tosh/status/2083593799872237680I'm not surprised that Claude Code is not optimized for an OpenAI model but I was still quite shocked re how much of a difference the harness makes.
Disclaimer: I'm working on 'smol' which is a minimalist harness but it's really nothing special, just a minimal system prompt, no skills files, only tool is shell
Do not underestimate how much popular harnesses are spamming the context window. The context window is very important.
Claude Code injects a ton of tools into the system prompt, including their “memory system” that’s like 10k+ tokens. Depending on your task shape, this can easily double your task cost (e.g. a low context-using job that takes many turns, like a monitoring loop).
You should use --disallowed-tools to prune any tools not needed for the task. Note that this is also a perpetual game of whack-a-mole since they’re always adding new tools.
> Claude Code injects a ton of tools into the system prompt
It's so unfortunate they don't let you use the subscription with other harnesses anymore - since even if I used OpenCode they'd still get a bunch of useful data from the API calls, meanwhile I could stretch their tier limits way further.
ty re --disallowed-tools
for coding agents 'shell' is often all you need (just make sure the environment has the necessary tools)
The system prompt part is surely cached across all users.
You still pay cache token costs on API calls. Cache cost/token are 90% lower, but you pay it every single turn.
I’m not sure if they let you skip the cache write cost on the first turn. That would imply cross-user caching infrastructure or special casing the default system prompt to give you a discount. Maybe? Away from the computer but you could try a “hello” in a fresh session and see what was billed.
For Anthropic models, yes. But OP was using Claude Code with GPT.
the system prompt still takes up useful space in the context window and steers the model into unnecessary actions and over-thinking patterns
What tools are you using to run the comparison? Or are you just running the prompts and checking them with something like ccusage?
I’m asking because I’ve been looking for agent harness comparison tools too. I’m interested in more than just the inputs and outputs—I also want the system prompts, traces, and tool calls. It’s useful to understand why Codex, for example, uses more tokens while Pi doesn’t.
Fewer tokens aren’t necessarily better if the agent skipped important checks. On the other hand, using more tokens could just mean it’s overthinking the process. Either way, seeing the full execution trace for the same task is really valuable.
I built an ad-hoc custom comparison framework to inspect system prompts, caching behaviour, tool call outputs, exact api requests and responses and so on
I agree fewer tokens is not necessarily better but a bit counter-intuitively often the harness using fewer tokens is not only done faster but has better results
(that said: of course check the results, look at the full traces, agree!)
Thanks! I ended up built my own too. Just thought there are other better options out there that I might've missed.
Running some local models and wondering what insanity was consuming 100k+ tokens to respond to "hello world" was rather eye-opening, yeah. Full of pointless fluff.
Anything else I can read about smol?
also, are you using a tool to collect those metrics? what is it?
smol is basically this 9 line python agent re-implemented in Go
https://news.ycombinator.com/item?id=49006862
I'll have more about it in the next hours/days, you can follow me on twitter in the meantime (https://x.com/__tosh)
Same task? How did the results compare?
the tasks were all simple agentic tasks
like creating a checksum of a file, merging csvs and so on, fixing a makefile pipeline
with known 'good' outcomes
all harnesses could reach the outcomes, only cost, time, number of tool uses and so on were different
(Claude Code failed once in 1 task but I think that was just an unfortunate outlier, the tasks aren't that difficult)
I was an early and passionate adopter and paying customer of Cursor (since 2023!), but it’s probably been 6 months since I opened it.
These days I “write” code with claude code and codex, and read/review it on GitHub. If I need to read it locally, I use a plain text editor.
Can someone help me understand what value cursor offers in 2026?
It removes money from your pocket into theirs faster than Claude will take money out of your pocket and into theirs. I was a Windsurf then Cursor user and even provided product feedback to the Cursor team on various aspects of their product but the costs for using it killed the value.
Cursor has two benefits
1) It's still an IDE. Because of pricing I mostly use Codex, but I always have a VSCode/Cursor IDE open, thus have to juggle between the two. Working directly in the IDE is more comfortable. For full on vibecoding that might be worse, but when you want to do a deep review of the changes, an IDE is way better than reading a diff on github.
2) It supports every model. It's often very helpful to try different models when you don't like the result of the first.
Can't VS Code extensions solve your use case?
I tried the codex extension many months ago but I didn't like it. I use mostly the Codex App, or sometimes the Codex TUI inside the IDE terminal.
The Claude plugin for VSC is amazing and basically "solves programming" for me. It's the primary way that I write code now; I even stopped using Claude Code.
Recently they also seem to have made big changes that makes Cursor worse for editing code by hand. I'm not sure what they are trying to do, but it doesn't feel like vscode fork anymore. I'm starting to consider alternatives.
To answer your question though, to me your workflow seems cumbersome. Cursor is more integrated and more frictionless.
I don't think it offers any. I got tired of switching between Codex and Claude plus I wanted to start using glm and kimi so I switch to pi. Then I got tired of VSCode eating up battery and RAM so I moved to Neovim which has barely any efficiency downside anymore since most code is not written by hand and if you know how to use it well it was only maybe 10% less efficient than VSCode to begin with. I am not sure why a heavy desktop application would be better than nvim + pi in terminal which are both super extensible via vibecoding and very light on resources.
Huh, never thought about that 2nd point - I'm also transitioning to pi, but didn't think to change to neovim as well. I guess your logic holds for claude code users as well though.
Cursor Tab is nice when you're hand-editing stuff. Not sure it's worth $20 though. I'd totally buy and forget a $5 Cursor subscription just for Cursor Tab.
The $20 price point is just too competitive now and I'd rather use the Claude/Codex plugin over Cursor's agentic coding sidebar.
The way they spin up multiple Composer 2.5 before sending it over to the model of your choice is nice.
How do you diff ?
Unless by "text editor" you mean Neovim or Emacs ?
Sounds like on GitHub. That's how I do it too.
git diff
Look at Cursor CLI: https://cursor.com/cli
Code and reasoning generated by Opus/Fable via Cursor seems to be quite better than Claude Code's for the same prompt.
Does that mean you’re paying api prices + cursor markup (?), or can you somehow piggyback on your Claude subscription? Fable via the API is really too expensive
Design mode: https://cursor.com/blog/design-mode
Cursor achieved fast adoption by making it simple to move from Visual Studio code.
Double-edged sword. It is also simple to move back to VS code and agent extensions.
This is basically what I did. In 2023 I moved from VS Code to Cursor. And in 2025 when Opus 4.7 (or was it 4.6?) did that big leap in December, I switched to Claude Code and VSCode for any edits. I come from Sublime Text so those shortcuts are hardwired in me (which was also the defaults in VSCode). When Cursor hijacked literally every CMD key combo, I got fed up and switched back to VSCode. Maybe I should just go back to Sublime Text now lol, since I really just need a blazing fast code reader.
Saw this at work yesterday. Absolute insanity to do something as user hostile as removing the cost of the service. There's no way to attempt to brand this as anything but negative for the user, and positive for the company.
Gotta justify a $60B purchase of an IDE and (at the time) a single, decent model.
This month I've noticed that my Cursor model use has gone up faster with Composer 2.5 - I have bothered with Grok yet. Despite the advertised twice-the-usage on Cursor own models I'd say it's burning faster.
Cursor has been my corporate vendor for accessing non-anthropic models. Without cost insights and with no ability to proxy API requests its value drops dramatically. They fought hard for a renewal with legacy pricing mode then broke the deal soon thereafter. I’ll be extremely clear to leadership that Cursor usage should be minimized and not renewed. I frankly don’t trust them with IP either, and you shouldn’t either, especially if you’re demonstrably not in a big enterprise.
Cursor was a great introduction to agentic engineering but I've learned their Claude pricing is largely just batch purchases. Their real moat I think is Composer 2.5 because both their agentic and IDE experiences fall short of Codex and Claude Desktop in my opinion. I think their sales will tell you economically they make the most sense and I would probably agree, but cost isn't everything especially when the spread isn't that wide. At this moment, capability is really king.
These days I'm using Codex and Claude Desktop with Zen when I need to look at code. Codex's real time audio chat feature (not dictation) is also second to none when paired with their agentic flow.
Grok 4.5 is them as well now I guess since they're owned by spacex now. I'd consider grok 4.5 the sonnet and composer the haiku (both capable, fast models).
It's free? Oh it's priced in "tokens".
Kind of (barely) like how Facebook has "friends". Or how a Snickers bar costs $1.99 and not $2.
Abstracting meaning out. Maximizing profits.
Enshittification.
If users start questioning the ROI of your AI product just hide the “I” from them. Problem solved.
They just don't want you feel obligated to thank them SO MUCH for hidden better prices
I’ve been working on a project to solve this problem with Claude code, codex, etc by recording usage and translating to $ in realtime. Looks like I’ll have to add Cursor support now.
Small thing but makes a big difference
The ground-truth is the rudeness is load bearing at human machine interface.
The signal is the ceiling.
I am shocked, shocked, that a company that is part of SpaceXAI would become hostile to its existing paying users and partners. This has never happened before in history.
cursor is a scam. they pissed me with their UI changes & shady pricing, so i tried claude code recently for the first time and instantly regreted i havent tried earlier
Built an awesome 450k line app with composer 2.5 and grok 4.5. I love it
Well ... don't use Cursor. It's really that simple.
Cursor you have a beautiful comeback story, you have a wonderful model with Composer 2.5, and a terrific behemoth with Grok 4.5 now. A top-tier dev ux with the Cursor agent app, why on earth are you squandering this opportunity and behaving this way?
You came back from the dead pretty much and now you're pissing it away for what exactly?
Do not spite your individual developer customers or you will perish yet again.
If they were smart, they'd rebrand Grok to Composer. Nobody wants to use the Nazi model.
Given that cursor has been bought by Musk, the nazi branding might be on purpose
please, you're dramatically overestimating who gives a shit about this.
The PM comment above matches what I've seen. Hiding a usage number doesn't remove the anxiety, it just removes the user's ability to verify it, so people pad their mental estimate upward and quietly disengage. For anything usage-billed, trust in the tool is capped by trust in the number, and you can't build that trust by making the number invisible.
The move that actually works is the opposite: make it legible. Show the breakdown, show what drove the spike, let people reconcile it against their own expectation. A scary number you can explain beats a hidden one they have to guess at.
Removing the CSV export is the part that stings most, because that's the one thing power users reach for the moment they stop trusting the dashboard.
Not a single one of these tools tell me how much I just spent when I issue a prompt and it finishes inference.
LLM generated comments are banned on this site.
Is AWS listening?