Does anyone have a suggestion for a harness that is good at handoff?
When I say handoff, I mean:
* handoff from a terminal CLI to webui (on a phone)?
* handoff from one team member, to another?
* handoff from one communication modality (like TUI) to email?
* handoff from one model to another
Does such a thing exist?
I used to think that a PR would be a good place to centralize all this. Who cares what IDE, or developer, or location. But, now I feel like an agent harness might contain that better.
I'm tempted to experiment with Pi to configure such a thing. But, perhaps there are patterns out there already with a harness I have not considered.
I’ve been working on a harness for accounting agents at my job recently and it’s been a pretty interesting experience.
We originally started with building a CLI tool so our LLMs could more easily interact with our platform. I cannot recommend enough the value of having an internal CLI. It’s both fun to build and extremely useful for agents.
We paired this with skills initially, but found that the way folks built skills was often too prescriptive and limited to the authors own specific function in the company. A 2k line long skill suffers from the same gaps as we do, if an agent is just following a laundry list it’s less likely to reason about the request it’s doing.
So we instead asked ourselves: what if we just _let_ the agent reason about the work to be done and only provided the tools + guardrails to gather context and perform accounting work?
Turns out frontier models are GOOD at what they do, they outperformed our highly prescriptive skills and were able to work across a larger set of tasks even without instruction on how to do those tasks.
It’s a breath of fresh air from the decade of CRUD I’ve worked on, harness engineering is very neat.
I’ve also found that Claude and friends are eerily good at using classic Unix CLI tools so I build mine in the same style, not unlike the `gh` CLI from GitHub, though with an agent-first design shape.
Usually I’m returning TSV as a default format and I add a `help-all` subcommand to list every available command at once when needed. Another thing that helps is adding just-in-time context-sensitive hints, such as: user has just run a list query with at least one result. Add a one-liner to the response explaining the command shape for getting the detail view of the first response.
In terms of skill files, I like to have my CLI generate them dynamically at runtime by walking their own current command tree and then feeding that through a text template.
Harnesses are the next frontier. If LLMs are electricity, harnesses are the “electronics.” Right now, it’s like an AC vs DC between Claude and ChatGPT, but once that settles, the harnesses will be the actual value providers.
And Pi is the best harness because of the amazing extension system. You can build extensions that turn Pi into a stock trader, software factory, anything. I tried switching to another harness but none have extension functionality as good as Pi.
Even if there is a new harness or agent project, I tell Pi to dig into the codebase and then make me an extension that brings that functionality into Pi. I did it with Prime Intellect’s and Deepseek’s harnesses and those are built on Pi.
I was saying more the custom skills and extensions that make the harness not a commodity. Yes people will use Claude Code, Codex, or Pi but their customizations will make their harness unique and more powerful.
In a sense they are the last frontier imo. At some point a harness will be built that can modify itself to fit the needs of the majority of people's workflows and evolve with them.
Then people will want to share and exchange their evolved harnesses. Ways will be found to modularize certain aspects to enable mixing and matching.
I’m thinking of how in cyberpunk, people are replacing their cybernetic enhancements all the time. You could alternatively bioengineer your own body towards the desired outcomes, but that’s constrained by the trajectory your body has already taken, whereas the promise of cybernetic parts is that they are more independently replaceable.
As another analogy, monolithic software tends to quickly become hard to change significantly, whereas a plugin architecture tends to be more flexible and modular, and people can share and combine their various plugins.
Sadly, many people have bought into the cult that LLMs will lead to AGI. I guess if that is your worldview then all this babbling about new frontiers makes more sense.
They probably used an LLM to come up with this bizarre metaphor.
Did I even mention AGI? All I’m saying is that we’re hitting a plateau with how good models are while harnesses are untapped potential. And with Pi, you can swap models like electricity companies. Yes for now, the electricity is better with some companies but this will stabilize.
And no I came up with the metaphor all on my own, send me the chat of you getting the LLM to come up with it. Why not argue based on merit instead of strawman and ad hominem attacks?
I’m afraid it’s a terrible metaphor, starting with the fact that LLMs are nothing like electricity, and the relation of harnesses to them is nothing like that of electronics to electricity, save perhaps one is a prerequisite of the other.
Harnesses (and the concept of agents before them) presuppose competence in LLMs which simply doesn’t exist.
“Just as electricity transformed almost everything 100 years ago, today I actually have a hard time thinking of an industry that I don’t think AI will transform in the next several years” - Andrew Ng
I didn’t come with the electricity idea, it was Sam Altman saying it will be like a utility down the line and metered[0]. What would the “electronics” be in your opinion?
> Sadly, many people have bought into the cult that LLMs will lead to AGI
You can never tell if the goomba opinion of the forum will agree we have reached AGI (seen that happen on a few threads lately) or will readily call that a ludicrous proposition.
I've never used Pi but I don't see why you can't use stock codex or claude code for the same purpose, what makes Pi special? I've built plenty of custom harnesses on top of claude code and codex using custom skills or simple markdown instructions and subagents. Never had any issues or limitations with that approach.
I do agree that harnesses are going to extend AI capabilities a lot in the next year, but after reading Pi's page I don't see anything that makes it particularly special in terms of functionality, other than being more provider-agnostic.
Codex and Claude historically had more bloat in their system prompt and tools. Pi is minimal by design so more adaptable. But to be fair Claude Code is moving in the Pi direction with a small system prompt.
Author here. I think our website could be much clearer - but Pi is fundamentally easier to mold than other harnesses. It’s not magic but it strikes the balance well of letting you shape it extensively without letting you break it.
A harness is the bottom layer of a pie that gets fed into the model. In my project, I count 7 more layers on top of it https://replicated.live/blog/wiki
They all affect consistency, coherence, token efficiency. Probably we need some broader term. Like "information architecture", "knowledge architecture"? It's not just shoveling Markdown to nvidias, after all.
For one you can ask Pi to create a TUI extension, so along with the agent interface you can add whatever custom TUI you need, such as portfolio stock tickers, alerts, whatever you want.
Many of my harnesses eventually turn into customized UIs around the chat interface.
This is a plug, but relevant. I recently added a 'build native tools on the fly' functionality to Dirac (https://github.com/dirac-run/dirac) that works like:
1. You can use the '/new-tool' and tell what kind of tool you want (including whether it should be task-scoped, workspace-scoped, or global), the model builds it, the harness runs validation and other tests until the tool is ready
2. The model decides that in such and such task, it would be helpful to have a tool like this, it can build a task-scoped tool.
In either scenario, the tool catalog is rebuilt, and the new tool is instantly available in the next turn.
This type of modification of the harness on the fly to fit the need is the future.
The only thing left after that is the mobile front. I think static app store type software as we know it is a thing of the past. You'll only ever need one self modifying app.
What I can see is a world where we end up with a Chromium-shaped harness, a fully featured standard implementation everyone builds against, because doing every single thing yourself would be crazy.
Pi doesn’t have a UI like Claude Desktop. It also doesn’t work with the Claude subscription, only API key and pricing.
So if you do want to use it, use the Codex sub. Once you install it, run Pi and /login and you’ll get login with ChatGPT. From there, Pi can tweak it’s settings if you ask. Check out their extensions (or ask Pi) and that will take you most of the way there.
I built a software factory and am now building a stock trader using opencandle extension[0] and a custom extension. For inspiration for how to tweak Pi, check out OMP, Prime Intellect, and Deepseek harnesses.
looking at the website. i can't really tell if they have benchmarks and measuremnts on how all that improves capablities over just using regular agent withtout all that
Yes but Pi has had a minimal system prompt since inception. Skills and Pi extensions let you make a hyper specific harness for specific use cases. For general conversation, harnesses are overkill most times.
There’s evidence of harnesses making a smaller, weaker model perform better than SOTA and some benchmarks ban harnesses because it becomes too easy.
Not a bad analogy because the bigger your backpack the slower you walk. With models a big context and tool set degrades performance. So you want the smallest harness/backpack that can do the job.
I have a similar mental model to the climbing harness. I think of LLMs as horses and harnesses as the saddle, reins, etc that you put on your horse. You might configure your harness for an individual rider or you might hook together several horses to pull a carriage.
To me, Before agentic programming a harness was like a mini framework in the app. Like for testing mostly. You’d set up the harness and configure it for your test and it would take care of boilerplate setup / optional reporting / benchmarks ect. Still works for both - but yea need a new word I guess
From a naming terms yes, the closest equivalent of the past is the term "framework". Having written a Go service framework that's how I perceived it and as I started to work on agents, anything related to that became an "agent harness". I guess naming and terms change with different paradigms.
A harness is "the code that gives a model an operational environment" according the the framing in https://github.com/shareAI-lab/learn-claude-code -- which is a build your own python claude code tutorial shareAI made after the code leak a few months back.
It's really funny (and a bit obnoxious) to watch the vocabulary from the outside. In 2023 everybody learned the word moat, then it's been agent(ic), from last year there's more talking about harnesses than at a bungee jumping convention. The mot du jour is frontier.
It truly proves like there's a handful of thought leaders on Twitter that everybody follows blindly and start to copy down to the lexicon and parrot everywhere else.
I thought this was the next evolution of the smartphone. One so smart that it does all the thinking for you. You don't even have to be conscious, you just do whatever it tells you too. Oh wait, that's what they do already.
Does anyone have a suggestion for a harness that is good at handoff?
When I say handoff, I mean:
Does such a thing exist?I used to think that a PR would be a good place to centralize all this. Who cares what IDE, or developer, or location. But, now I feel like an agent harness might contain that better.
I'm tempted to experiment with Pi to configure such a thing. But, perhaps there are patterns out there already with a harness I have not considered.
I’ve been working on a harness for accounting agents at my job recently and it’s been a pretty interesting experience.
We originally started with building a CLI tool so our LLMs could more easily interact with our platform. I cannot recommend enough the value of having an internal CLI. It’s both fun to build and extremely useful for agents.
We paired this with skills initially, but found that the way folks built skills was often too prescriptive and limited to the authors own specific function in the company. A 2k line long skill suffers from the same gaps as we do, if an agent is just following a laundry list it’s less likely to reason about the request it’s doing.
So we instead asked ourselves: what if we just _let_ the agent reason about the work to be done and only provided the tools + guardrails to gather context and perform accounting work?
Turns out frontier models are GOOD at what they do, they outperformed our highly prescriptive skills and were able to work across a larger set of tasks even without instruction on how to do those tasks.
It’s a breath of fresh air from the decade of CRUD I’ve worked on, harness engineering is very neat.
Can you post a generic version of code for this somewhere (e.g. codeberg or whatever)?
I find your description intriguing but I'd like to see it to make sure I understand it.
So you still have CLIs but they have I presume an help command that describes the capabilities right.
Could you give an example of an accounting guardrail you created?
I’ve also found that Claude and friends are eerily good at using classic Unix CLI tools so I build mine in the same style, not unlike the `gh` CLI from GitHub, though with an agent-first design shape.
Usually I’m returning TSV as a default format and I add a `help-all` subcommand to list every available command at once when needed. Another thing that helps is adding just-in-time context-sensitive hints, such as: user has just run a list query with at least one result. Add a one-liner to the response explaining the command shape for getting the detail view of the first response.
In terms of skill files, I like to have my CLI generate them dynamically at runtime by walking their own current command tree and then feeding that through a text template.
Examples from a public project: https://github.com/radiusmethod/gitlab-kiosk/blob/main/skill...
Harnesses are the next frontier. If LLMs are electricity, harnesses are the “electronics.” Right now, it’s like an AC vs DC between Claude and ChatGPT, but once that settles, the harnesses will be the actual value providers.
And Pi is the best harness because of the amazing extension system. You can build extensions that turn Pi into a stock trader, software factory, anything. I tried switching to another harness but none have extension functionality as good as Pi.
Even if there is a new harness or agent project, I tell Pi to dig into the codebase and then make me an extension that brings that functionality into Pi. I did it with Prime Intellect’s and Deepseek’s harnesses and those are built on Pi.
> If LLMs are electricity, harnesses are the “electronics.” (...) the harnesses will be the actual value providers.
Don't get ahead of yourself. Harnesses are not exactly rocket science and will be a commodity.
The real value providers here are the hardware, then the LLM as a distant second, and at a much larger distance the harness.
https://www.latent.space/p/attention-interface
Labs are now post-training models with Harness so that Harness now gets absorbed into the weights.
I’d say that harnesses almost by definition are the parts that you want to keep customizable. That won’t get absorbed into the weights.
My naive intuition is that as harnesses converge on shape and models improve the first party advantage will mostly disappear.
Either part can be branded a "commodity" or a "sovereign privilege" depending on supply and demand.
Solar goes all the way up => power is commodity.
Some hyperscaler goes bankrupt => hardware is commodity.
Models get real good => output is a commodity, no profitable problems to solve anymore.
Open source models get good => models are commodity.
I was saying more the custom skills and extensions that make the harness not a commodity. Yes people will use Claude Code, Codex, or Pi but their customizations will make their harness unique and more powerful.
> Harnesses are the next frontier. If LLMs are electricity, harnesses are the “electronics.”
I really though this comment was a satire ...
In a sense they are the last frontier imo. At some point a harness will be built that can modify itself to fit the needs of the majority of people's workflows and evolve with them.
Then people will want to share and exchange their evolved harnesses. Ways will be found to modularize certain aspects to enable mixing and matching.
I’m thinking of how in cyberpunk, people are replacing their cybernetic enhancements all the time. You could alternatively bioengineer your own body towards the desired outcomes, but that’s constrained by the trajectory your body has already taken, whereas the promise of cybernetic parts is that they are more independently replaceable.
As another analogy, monolithic software tends to quickly become hard to change significantly, whereas a plugin architecture tends to be more flexible and modular, and people can share and combine their various plugins.
E = mc^2 + AI
Sadly, many people have bought into the cult that LLMs will lead to AGI. I guess if that is your worldview then all this babbling about new frontiers makes more sense.
They probably used an LLM to come up with this bizarre metaphor.
Did I even mention AGI? All I’m saying is that we’re hitting a plateau with how good models are while harnesses are untapped potential. And with Pi, you can swap models like electricity companies. Yes for now, the electricity is better with some companies but this will stabilize.
And no I came up with the metaphor all on my own, send me the chat of you getting the LLM to come up with it. Why not argue based on merit instead of strawman and ad hominem attacks?
I’m afraid it’s a terrible metaphor, starting with the fact that LLMs are nothing like electricity, and the relation of harnesses to them is nothing like that of electronics to electricity, save perhaps one is a prerequisite of the other.
Harnesses (and the concept of agents before them) presuppose competence in LLMs which simply doesn’t exist.
“Just as electricity transformed almost everything 100 years ago, today I actually have a hard time thinking of an industry that I don’t think AI will transform in the next several years” - Andrew Ng
Is it just a coincidence that he works in the space and will directly benefit if this is true?
I didn’t come with the electricity idea, it was Sam Altman saying it will be like a utility down the line and metered[0]. What would the “electronics” be in your opinion?
0. https://www.businessinsider.com/sam-altman-ai-utility-electr...
Altman is a salesman selling flimflam to people who should know better.
His idea of metering is predicated on the thing he’s selling being AGI.
> Sadly, many people have bought into the cult that LLMs will lead to AGI
You can never tell if the goomba opinion of the forum will agree we have reached AGI (seen that happen on a few threads lately) or will readily call that a ludicrous proposition.
I've never used Pi but I don't see why you can't use stock codex or claude code for the same purpose, what makes Pi special? I've built plenty of custom harnesses on top of claude code and codex using custom skills or simple markdown instructions and subagents. Never had any issues or limitations with that approach.
I do agree that harnesses are going to extend AI capabilities a lot in the next year, but after reading Pi's page I don't see anything that makes it particularly special in terms of functionality, other than being more provider-agnostic.
Codex and Claude historically had more bloat in their system prompt and tools. Pi is minimal by design so more adaptable. But to be fair Claude Code is moving in the Pi direction with a small system prompt.
Author here. I think our website could be much clearer - but Pi is fundamentally easier to mold than other harnesses. It’s not magic but it strikes the balance well of letting you shape it extensively without letting you break it.
A harness is the bottom layer of a pie that gets fed into the model. In my project, I count 7 more layers on top of it https://replicated.live/blog/wiki They all affect consistency, coherence, token efficiency. Probably we need some broader term. Like "information architecture", "knowledge architecture"? It's not just shoveling Markdown to nvidias, after all.
For one you can ask Pi to create a TUI extension, so along with the agent interface you can add whatever custom TUI you need, such as portfolio stock tickers, alerts, whatever you want.
Many of my harnesses eventually turn into customized UIs around the chat interface.
If LLMs are oxen, harnesses are... the harnesses
Yeah. This is pretty clearly the origin of the usage.
The harness facilitates the work animal doing work for you.
Not climbing harnesses to keep you safe.
> ...once that settles, the harnesses will be the actual value providers.
The words "once that settles" are doing historic levels of work here.
No human on earth has a clear idea whether model technology will settle tomorrow or 100 years from now.
There's every reason to expect architectural breakthroughs will keep being discovered and causing nuclear blasts of forward progress.
Both Claude and codex are unappealing, crap, generic agents that you have 0 control over.
Don't understand what people see in them.
This is a plug, but relevant. I recently added a 'build native tools on the fly' functionality to Dirac (https://github.com/dirac-run/dirac) that works like:
1. You can use the '/new-tool' and tell what kind of tool you want (including whether it should be task-scoped, workspace-scoped, or global), the model builds it, the harness runs validation and other tests until the tool is ready
2. The model decides that in such and such task, it would be helpful to have a tool like this, it can build a task-scoped tool.
In either scenario, the tool catalog is rebuilt, and the new tool is instantly available in the next turn.
This type of modification of the harness on the fly to fit the need is the future. The only thing left after that is the mobile front. I think static app store type software as we know it is a thing of the past. You'll only ever need one self modifying app.
I don't think so.
What I can see is a world where we end up with a Chromium-shaped harness, a fully featured standard implementation everyone builds against, because doing every single thing yourself would be crazy.
The antithesis to Pi, if you will.
What did you bring over from prime-agent? (I use prime-agent as my daily since it launched)
I primarily like how it manages sessions, and how agents can easily reference other sessions.
Something like this since Prime Intellect uses RLM under the hood:
https://github.com/manojlds/pi-rlm
Pi's most popular extensions, by download count:
https://pi.dev/packages?type=extension
I want to move from Claude Desktop to Pi, but I found it a little unfriendly. Any tips to set it up?
Pi doesn’t have a UI like Claude Desktop. It also doesn’t work with the Claude subscription, only API key and pricing.
So if you do want to use it, use the Codex sub. Once you install it, run Pi and /login and you’ll get login with ChatGPT. From there, Pi can tweak it’s settings if you ask. Check out their extensions (or ask Pi) and that will take you most of the way there.
What hiccups were you having?
> It also doesn’t work with the Claude subscription, only API key and pricing.
Not out of the box, but you can add agent sdk. I'm not sure how great the results will be though.
I haven’t tried it myself yet but I’m under the impression that Hermes Agent might be what you’re looking for?
Can you be more specific regarding unfriendliness?
Please tell me this is satire, it reads like straight from the depths of LinkedIn where a while loop is seen as the second coming…
Are human HN commenters now starting to speak in a dialect of Claudish?
The harness is just another codebase for the model to write and optimize. The value is still very much in the model.
How does Pi compare to vscode? Admittedly that is the only “agent/harness” I’ve ever used.
what have you built other than a harness?
I built a software factory and am now building a stock trader using opencandle extension[0] and a custom extension. For inspiration for how to tweak Pi, check out OMP, Prime Intellect, and Deepseek harnesses.
0. https://github.com/Kahtaf/OpenCandle
> Prime Intellect
looking at the website. i can't really tell if they have benchmarks and measuremnts on how all that improves capablities over just using regular agent withtout all that
i think its the opposite. claude code apparently removed hundreds of lines of system prompt because its not relavent anymore with newer models.
also i think its hard to build general harnesses if they were trained on specific harness architecture.
Yes but Pi has had a minimal system prompt since inception. Skills and Pi extensions let you make a hyper specific harness for specific use cases. For general conversation, harnesses are overkill most times.
There’s evidence of harnesses making a smaller, weaker model perform better than SOTA and some benchmarks ban harnesses because it becomes too easy.
All agentic editors/frameworks have skills and extensions and plugins?
Clear, relevant, and easy to understand. Thank you for writing this up, I’ll be sharing this link with all my non-tech friends!
Author here. Thanks - I appreciate the feedback
i also like the backpack analogy
the harness is what you take with you on a trip/task
whatever you take with you is not free (system prompt, tools, skills …)
some models are really good even if you bring almost no skills, tools or system prompt
the harness is the complement to the model
the better the model the more minimal the harness can be
harnesses like pi [0] and smol [1]are on the more minimal end of things
[0] https://github.com/earendil-works/pi
[1] https://github.com/smol-env/smol
Not a bad analogy because the bigger your backpack the slower you walk. With models a big context and tool set degrades performance. So you want the smallest harness/backpack that can do the job.
I have a similar mental model to the climbing harness. I think of LLMs as horses and harnesses as the saddle, reins, etc that you put on your horse. You might configure your harness for an individual rider or you might hook together several horses to pull a carriage.
To me, Before agentic programming a harness was like a mini framework in the app. Like for testing mostly. You’d set up the harness and configure it for your test and it would take care of boilerplate setup / optional reporting / benchmarks ect. Still works for both - but yea need a new word I guess
From a naming terms yes, the closest equivalent of the past is the term "framework". Having written a Go service framework that's how I perceived it and as I started to work on agents, anything related to that became an "agent harness". I guess naming and terms change with different paradigms.
A harness is "the code that gives a model an operational environment" according the the framing in https://github.com/shareAI-lab/learn-claude-code -- which is a build your own python claude code tutorial shareAI made after the code leak a few months back.
The ai hype word for 2026 after agent in 2025 for any LLM powered application.
Well kind of, I wouldn't be surprised to see that some things marketed as agents are actually good old deterministic software.
It's really funny (and a bit obnoxious) to watch the vocabulary from the outside. In 2023 everybody learned the word moat, then it's been agent(ic), from last year there's more talking about harnesses than at a bungee jumping convention. The mot du jour is frontier.
It truly proves like there's a handful of thought leaders on Twitter that everybody follows blindly and start to copy down to the lexicon and parrot everywhere else.
Is harness load-bearing?
Another AI harness is Goose, which is OSS and housed under the Linux Foundation (LF) Agentic AI Foundation (AAIF):
https://github.com/aaif-goose/goose
Full disclosure: I work at the LF, but not the AAIF.
Harnesses are hands.
Written using a harness? Too verbose to be read.
I thought this was the next evolution of the smartphone. One so smart that it does all the thinking for you. You don't even have to be conscious, you just do whatever it tells you too. Oh wait, that's what they do already.