We built Stagehand 2 years ago (24k stars and 4M monthly npm downloads) and recently fixed its biggest flaw: round-trip latency.
Every action performed requires a round trip between your script and the browser (short when running locally but increased when running in the cloud). We also saw multiple posts complaining about the eager token appetite of Playwright MCP.
For this reason, we rebuilt Stagehand from the ground up and shipped v4, where Stagehand controls the browser from an extension automatically loaded upon your browser startup.
Stagehand v4 comes with batch command support, dedicated token-efficient methods `act()` and `extract()`, and a brand new architecture making it 2x faster than Playwright and 80% more token efficient.
You can see for yourself by looking at our benchmarks, comparing its performance across a dozen models (frontier and open weights) and tools (Codex, Claude Code, and more): https://www.stagehand.dev/evals
I've been using Playwright a lot and also hit on the friction between playwright's web testing scope (e.g. no 1st class support for "muted" start-up of browsers, jeez) and usage for task automation all the time. This looks very interesting!
How does stagehand deal with complex http/websocket request/response and or console message filtering? https://docs.stagehand.dev/v4/reference/page#on E.g. I would like a script that tracks all communication that matches a specific filter (implemented as an anonymous function/lambda). This filter may look at patterns in the url, but sometimes needs to do a deeper inspection of also the payload (if the url does not carry enough information in itself).
I've found playwright to be prohibitively slow at this, not only because of the round-trip latency, but just the simple fact that it needs to pump the complete response to my filter function, which then proceeds to read only a couple of bytes to make the filtering decision. There are a lot of cases where I am only interested in around 1% of the total requests processed by the filter, which makes this behaviour massively wasteful.
Ideally I would like to run this filter in the browser as well. It currently simply searches the first 100 bytes (usually enough) for a given substring, but a more flexible filter would be good, perhaps even a filter func that is eval'ed in the extension? From the documentation, I don't see this use-case is currently supported. Are there any plans along these lines? :)
I'm using playwright to do diagnosis on customer sites, and I'm wondering some things:
1) Does it have full Chrome Devtools support? Like, able to diagnose where a fetch() came from, what script sets a cookie?
2) How hard would it be to set up in an AWS Lambda (I can run Playwright without needing to run a container, which saves a lot of startup time/cost)?
Always interesting to see forward work on these kinds of tools which get used so foundationally billions of times a day.
Looks very useful, and I like the caching idea which I think makes it interesting for self-healing CI tests.
How does it determine when a cached act() fails and has to be re-evaluated by the LLM? And in particular, if the cache is saved in the cloud (Browserbase?), won't this lead to a lot of cache churn if used in CI pipelines where different versions of the site are running against the same cache?
Also, is there a technical reason why the cache couldn't just be a local file that's checked in along with the script but must be provided by Browserbase? If it was, devs could heal failing tests locally using LLM calls, while CI runs entirely deterministically.
The cache serves as a managed server-side layer in the Stagehand API, keyed to instruction, page content, and the options you provide. Model configuration is intentionally excluded from the cache, so switching models does not invalidate it. If the page structure changes, the call does not result in a cache hit, and we revert to full inference. Additionally, any cached selector that no longer resolves will also revert, with self-healing disabled during replay.
Regarding the local file issue: currently, caching keys are based on a Browserbase session. This means that with a purely local browser, the cache option has no effect, and there’s nothing to verify. Each call returns metadata (including status, miss reason, threshold, count, and tokens saved) so you can observe churn rather than make guesses about it.
Exactly, as Stagehand now runs inside the browser, you'll save on the round trip.
Also, enabling batch actions will further accelerate your test suite.
Does "agent-based" here mean a CLI/Coding agent or software-level agents you built?
I'd say that Stagehand is subtly faster for local use cases (batching, in-extension design) while covering both CLI (via https://browse.sh) and framework-level use, whereas agent-browser is CLI-only.
It still uses CDP but communicates from an extension within the browser instead of a script running in a separate runtime or, worse, in a separate region.
Where would the control come from? Software installed on the client's computer, or software installed connecting to an AI agent, or something else, something more external, something more interactive?
Yes, we install a small piece of software on their computers because we also separately control GUI of legacy windows apps. But we also need to control web based apps as well.
I think what they are suggesting is that this would replace playwright/cypress in favor of stagehand+test runner (like vitest or whatever you are already using)
seems really cool! although, one question i have is why not keep agent() alongside the new primatives? it seems v4 removed agent() entirely rather than offering it with all of act(), observe(), extract().
We removed agent because so many great harnesses are available in the ecosystem.
Instead of keeping it, we decided to make Stagehand v4 better integrated with popular harnesses, both at the Coding Agent level (Codex, Claude Code) and frameworks level (Eve, Deep Agents, Mastra, etc)
A lot, especially for performance. That's why we built our own benchmarks that account for both the model and the harness.
For example, with Claude Opus 5, the accuracy gap can be up to 3% and performance up to 200ms, depending on whether you're using Deep Agents, Eve, or Fx.
I tried this myself - my conclusion was the overhead of parsing a screenshot, generating an action, and being limited to headful mode is much less efficient than reading accessibility trees & generating CDP commands.
basically, browser automation is a closer-to-the-metal abstraction than computer use, allows more flexibility, and ends up being much cheaper at scale!
If agents need to run on the desktop or logged into a user's real browser, the alternative would be Ui.Vision MCP. But stagehand is designed for backend use, so apples and oranges.
We built Stagehand 2 years ago (24k stars and 4M monthly npm downloads) and recently fixed its biggest flaw: round-trip latency.
Every action performed requires a round trip between your script and the browser (short when running locally but increased when running in the cloud). We also saw multiple posts complaining about the eager token appetite of Playwright MCP.
For this reason, we rebuilt Stagehand from the ground up and shipped v4, where Stagehand controls the browser from an extension automatically loaded upon your browser startup.
Stagehand v4 comes with batch command support, dedicated token-efficient methods `act()` and `extract()`, and a brand new architecture making it 2x faster than Playwright and 80% more token efficient.
You can see for yourself by looking at our benchmarks, comparing its performance across a dozen models (frontier and open weights) and tools (Codex, Claude Code, and more): https://www.stagehand.dev/evals
Ask me anything!
I've been using Playwright a lot and also hit on the friction between playwright's web testing scope (e.g. no 1st class support for "muted" start-up of browsers, jeez) and usage for task automation all the time. This looks very interesting!
How does stagehand deal with complex http/websocket request/response and or console message filtering? https://docs.stagehand.dev/v4/reference/page#on E.g. I would like a script that tracks all communication that matches a specific filter (implemented as an anonymous function/lambda). This filter may look at patterns in the url, but sometimes needs to do a deeper inspection of also the payload (if the url does not carry enough information in itself).
I've found playwright to be prohibitively slow at this, not only because of the round-trip latency, but just the simple fact that it needs to pump the complete response to my filter function, which then proceeds to read only a couple of bytes to make the filtering decision. There are a lot of cases where I am only interested in around 1% of the total requests processed by the filter, which makes this behaviour massively wasteful.
Ideally I would like to run this filter in the browser as well. It currently simply searches the first 100 bytes (usually enough) for a given substring, but a more flexible filter would be good, perhaps even a filter func that is eval'ed in the extension? From the documentation, I don't see this use-case is currently supported. Are there any plans along these lines? :)
I'm confused. You flagged your own replies?
[flagged]
I'm using playwright to do diagnosis on customer sites, and I'm wondering some things:
1) Does it have full Chrome Devtools support? Like, able to diagnose where a fetch() came from, what script sets a cookie? 2) How hard would it be to set up in an AWS Lambda (I can run Playwright without needing to run a container, which saves a lot of startup time/cost)?
Always interesting to see forward work on these kinds of tools which get used so foundationally billions of times a day.
[dead]
How does the “Token efficiency Hybrid accessibility-tree trimming” work?
[dead]
We also just added Jev support https://github.com/browserbase/stagehand/pull/2952
On the act/extract/observe evals it shows promising results being extremely efficient
- act: 4.3x faster, 97% fewer LLM calls. Pass rate: 97.5% -> 98.3%. - heldout: 4.1x faster, 78% fewer LLM calls. Pass rate: 87.5% -> 97.5%. - observe: 11.1x faster, 69% fewer LLM calls. Pass rate: 75.0% -> 83.3%. - extract: 8.7x faster, 75% fewer LLM calls. Pass rate unchanged at 92%.
cost effectively 0
Looks very useful, and I like the caching idea which I think makes it interesting for self-healing CI tests.
How does it determine when a cached act() fails and has to be re-evaluated by the LLM? And in particular, if the cache is saved in the cloud (Browserbase?), won't this lead to a lot of cache churn if used in CI pipelines where different versions of the site are running against the same cache?
Also, is there a technical reason why the cache couldn't just be a local file that's checked in along with the script but must be provided by Browserbase? If it was, devs could heal failing tests locally using LLM calls, while CI runs entirely deterministically.
The cache serves as a managed server-side layer in the Stagehand API, keyed to instruction, page content, and the options you provide. Model configuration is intentionally excluded from the cache, so switching models does not invalidate it. If the page structure changes, the call does not result in a cache hit, and we revert to full inference. Additionally, any cached selector that no longer resolves will also revert, with self-healing disabled during replay.
Regarding the local file issue: currently, caching keys are based on a Browserbase session. This means that with a purely local browser, the cache option has no effect, and there’s nothing to verify. Each call returns metadata (including status, miss reason, threshold, count, and tokens saved) so you can observe churn rather than make guesses about it.
So for an enterprise with a 1,000 test Playwright suite, does this basically mean ~2x faster CI times? That would be huge.
Exactly, as Stagehand now runs inside the browser, you'll save on the round trip. Also, enabling batch actions will further accelerate your test suite.
Tests typically drive the browser on the same machine. What round trip are we talking about here? Process-to-process over CDP?
[dead]
judging from their documentation looks like promised increase only achievable with their main product
Nope, you can also deploy and manage your own browsers as soon as they accept custom extensions.
Would you recommend to use this over agent-browser for general agent-based validation work? Any performance benefit?
Does "agent-based" here mean a CLI/Coding agent or software-level agents you built? I'd say that Stagehand is subtly faster for local use cases (batching, in-extension design) while covering both CLI (via https://browse.sh) and framework-level use, whereas agent-browser is CLI-only.
what did you lose compared to CDP (e.g. cross-origin iframes, downloads running in envs where you can't load extensions)?
It still uses CDP but communicates from an extension within the browser instead of a script running in a separate runtime or, worse, in a separate region.
Are you using CDP over web sockets or via a pipe?
[dead]
If your needs are simpler, I created a tiny headless WebKit browser specifically for agents called Sleepy Hollow [0]
[0]: https://github.com/bensyverson/sleepyhollow
What would be the Linux equivalent?
WebKit is available on Linux, just not Apple’s wrapper. It would certainly be possible to give Sleepy a Linux backend if you’re motivated!
I love the name.
Can this be used for browser automation on customer computers? The only glaring issue I see is the api tokens are exposed for LLMs.
Where would the control come from? Software installed on the client's computer, or software installed connecting to an AI agent, or something else, something more external, something more interactive?
Yes, we install a small piece of software on their computers because we also separately control GUI of legacy windows apps. But we also need to control web based apps as well.
Is there something like this for cypress?
I think what they are suggesting is that this would replace playwright/cypress in favor of stagehand+test runner (like vitest or whatever you are already using)
seems really cool! although, one question i have is why not keep agent() alongside the new primatives? it seems v4 removed agent() entirely rather than offering it with all of act(), observe(), extract().
We removed agent because so many great harnesses are available in the ecosystem. Instead of keeping it, we decided to make Stagehand v4 better integrated with popular harnesses, both at the Coding Agent level (Codex, Claude Code) and frameworks level (Eve, Deep Agents, Mastra, etc)
Do I need to pay for Browserbase to use this?
Nope, Stagehand is open-source and works with local browsers by default.
[flagged]
How much does the harness really matter for evals?
A lot, especially for performance. That's why we built our own benchmarks that account for both the model and the harness. For example, with Claude Opus 5, the accuracy gap can be up to 3% and performance up to 200ms, depending on whether you're using Deep Agents, Eve, or Fx.
More details here: https://www.stagehand.dev/evals
lowk why not just do astra computer use
I tried this myself - my conclusion was the overhead of parsing a screenshot, generating an action, and being limited to headful mode is much less efficient than reading accessibility trees & generating CDP commands.
basically, browser automation is a closer-to-the-metal abstraction than computer use, allows more flexibility, and ends up being much cheaper at scale!
Astra with computer use seems to burn a ton of tokens though. Stagehand seems more token efficient.
If agents need to run on the desktop or logged into a user's real browser, the alternative would be Ui.Vision MCP. But stagehand is designed for backend use, so apples and oranges.
Money?
[flagged]
[dead]
no matter how much faster you make.. playwright is playwright. Dead bot giveaway.
A lot of folks use Playwright for UI testing.