# why
Evals should be able to run without the braintrust dependency
# what changed
- Added `hasBraintrustApiKey()` and made `tracedSpan()` a passthrough
when no key.
- `AISdkClientWrapped` falls back to the plain `ai` SDK when
`braintrust` wrapping isn’t available; introduced a `WrappedAI` type and
lazy load keyed on `BRAINTRUST_API_KEY`.
# test plan
- [x] Added tests for key-present/absent paths and updated `core-runner`
test to set a dummy key; fixes for test stability and build.
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Make Braintrust optional for evals. Evals run without
`BRAINTRUST_API_KEY` and skip sending logs; behavior is unchanged when a
key is set.
- **Bug Fixes**
- Added `hasBraintrustApiKey()` and made `tracedSpan()` a passthrough
when no key.
- Runner now passes `noSendLogs: true` to `Eval` and skips `flush()`
without a key; sends logs and flushes when present.
- `AISdkClientWrapped` falls back to the plain `ai` SDK when
`braintrust` wrapping isn’t available; introduced a `WrappedAI` type and
lazy load keyed on `BRAINTRUST_API_KEY`.
- Added tests for key-present/absent paths and updated `core-runner`
test to set a dummy key; fixes for test stability and build.
<sup>Written for commit 482e49bac6dce88ea110aa9a4f06a50346617a06.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/2153?utm_source=github">Review
in cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
---------
Co-authored-by: Chromie <miguel@browserbase.com>
Co-authored-by: Claude <noreply@anthropic.com>
# Why
The verifier rewrite needs to coexist with the legacy v3 evaluator so
existing evals and callers do not silently change behavior while the new
pipeline is reviewed. This PR creates the compatibility boundary that
lets us select the evaluator backend explicitly.
# What Changed
- Extracted the current v3 evaluator behavior into `LegacyV3Evaluator`.
- Kept `V3Evaluator` as the public facade.
- Added `STAGEHAND_EVALUATOR_BACKEND=legacy|verifier` and constructor
backend options.
- Defaulted the backend to `legacy` to preserve current `ask()` and
`batchAsk()` semantics.
- Added public API tests for backend selection and invalid backend
handling.
# Tests
- `pnpm --filter @browserbasehq/stagehand run typecheck`
- `pnpm --filter @browserbasehq/stagehand run test:core --
packages/core/dist/esm/tests/unit/public-api/v3-core.test.js`
- `git diff --check`
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
# why
Some pages do not properly encapsulate the necessary information
required to extract the content requested by the user, and can benefit
from a hybrid approach with vision.
# what changed
- New `extract({ options: { screenshot: true } })` flag. Captures the
current viewport as PNG and sends it as an `image_url` part alongside
the a11y tree text in the extraction LLM call.
- Default is `false` — existing callers see no behavior change.
- Gated to AI SDK clients; throws `StagehandInvalidArgumentError`
otherwise (validated in both the handler and `inference.extract`, with
the handler check running before the screenshot is taken).
- Screenshot capture is bracketed by `ensureTimeRemaining()` so it
respects the extract timeout.
- Wired end-to-end: public `ExtractOptions` type,
`ExtractOptionsSchema`, OpenAPI `ExtractOptions`,
`StagehandAPIClient.extract` wire payload, `V3.extract` →
`ExtractHandler` → `inference.extract` →
`buildExtract{System,User}Prompt`.
# test plan
- `extract-screenshot.test.ts` (new) — multimodal message shape on happy
path; rejection for non-AI SDK clients.
- `timeout-handlers.test.ts` — handler captures viewport when enabled,
skips capture by default, rejects screenshot + non-AI SDK client without
touching `page.screenshot` or the snapshot pipeline.
- `api-client-serialization.test.ts` (renamed from
`api-client-observe-variables.test.ts`) — wire payload preserves
`options.screenshot`.
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Adds a screenshot option to `extract()` that sends the current viewport
screenshot with the accessibility tree to improve results on
visually-driven pages.
- New Features
- `extract({ options: { screenshot: boolean } })` captures a PNG of the
current viewport and includes it with the accessibility tree; defaults
to false; only supported with `aisdk` clients (throws
`StagehandInvalidArgumentError` otherwise).
- Prompts are consolidated and multimodal when enabled: system and user
prompts mention the screenshot and using it with the accessibility tree;
attaches an `image_url` data URI; logs when a screenshot is used.
- Public API, types, and OpenAPI updated (`ExtractOptions.screenshot`);
the API client forwards the flag; tests cover prompt shape, viewport
capture, default-off behavior, timeout guards, client-type validation,
and serialization.
<sup>Written for commit d0c75002884eb42fcebb132c0d333259e8d9efbf.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/2149?utm_source=github">Review
in cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
## Why
The generated SDK/OpenAPI surface needs to expose the Vertex
service-account auth fields that Stagehand can already pass through.
## What changed
- Add `vertex` model config fields to the canonical Stagehand Zod
schema: `project`, `location`, `baseURL`, and `googleAuthOptions`.
- Model `googleAuthOptions.credentials` as a strict
service-account-compatible object requiring `client_email` and
`private_key`.
- Omit file-path auth, external credential-source auth, and Vertex
Express Mode API-key auth from the AI SDK v5-compatible schema.
- Regenerate the v3 OpenAPI spec from the Zod source of truth.
- Add schema tests covering Vertex auth passthrough, minimal
credentials, and rejection of unsupported auth shapes.
---------
Co-authored-by: samfinton <samfinton@samLaptopm5.galaxus.box>
Co-authored-by: samfinton <samfinton@samLaptopm5.local>
## Summary
- Pass `localBrowserLaunchOptions` into `V3Context.create()` when LOCAL
mode attaches to an existing browser via `cdpUrl`.
- Add a regression test that verifies the CDP attach path forwards
viewport, device scale factor, and CDP headers through context creation.
- Add a patch changeset for the published Stagehand runtime behavior
change.
## Why
The launch path already passes local browser options into `V3Context`,
which lets `Page.create()` apply the configured viewport and
`deviceScaleFactor`. The CDP attach path only forwarded `env` and
`cdpHeaders`, so attached local sessions skipped that Page-level
viewport setup.
## E2E Test Matrix
| Command / flow | Observed output | Confidence / sufficiency |
| --- | --- | --- |
| `pnpm install --frozen-lockfile` | Installed all 7 workspace projects
from the lockfile; completed with `Done in 13.9s`. | Confirms the fresh
PR worktree can resolve dependencies exactly from the committed
lockfile. |
| `pnpm exec turbo run build:esm --filter=@browserbasehq/stagehand` |
Turbo ran `gen-version`, `build-dom-scripts`, and `build:esm`; output
ended with `Tasks: 3 successful, 3 total`. | Builds the exact local
package under review, including generated runtime artifacts needed by
the ESM output. |
| `pnpm --filter @browserbasehq/stagehand exec vitest run --config
vitest.esm.config.mjs
dist/esm/tests/unit/browserbase-session-accessors.test.js` | Vitest
output: `Test Files 1 passed (1)`, `Tests 6 passed (6)`. | Confirms the
regression test and nearby Browserbase/LOCAL accessor tests pass against
the built ESM output. |
| `node --input-type=module <inline LOCAL CDP attach viewport smoke>` |
Launched local Stagehand, reattached via `cdpUrl` with `viewport: {
width: 1440, height: 900 }`, and printed
`{"status":"pass","cdpAttachViewport":{"innerWidth":1440,"innerHeight":900,"devicePixelRatio":1,"bottomRightHit":"target"}}`.
| Exercises the real built Stagehand package against a real local Chrome
CDP session. Proves explicit CDP attach viewport settings control
browser CSS viewport and bottom-right coordinate hit-testing; does not
exercise kernel.sh specifically. |
| `pnpm exec changeset status --since=origin/main` | Changesets reported
patch bumps for `@browserbasehq/stagehand` and dependent workspace
packages. | Confirms the new changeset is detected as a
release-triggering patch for this runtime behavior fix. |
---------
Co-authored-by: Shrey Pandya <shrey@browserbase.com>
# why
- the cli has moved to a different repo
# what changed
- removed release-cli.yml & changelog.md
- cleaned up cli publish related metadata from package.json
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Remove the CLI release workflow and publish hooks now that
`@browserbasehq/browse-cli` lives in its own repo. This prevents
accidental publishes from this monorepo and simplifies releases.
- **Refactors**
- Deleted `.github/workflows/release-cli.yml` and removed CLI
publish/canary steps from `.github/workflows/release.yml`.
- Removed `packages/cli/CHANGELOG.md`.
- Cleaned up publish-related fields in `packages/cli/package.json`
(publishConfig, files, prepublishOnly).
- Cleared `.changeset/config.json` ignore list to stop special-casing
the CLI.
<sup>Written for commit 880c857cf93643500165ed722e29e9ca69218aa9.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/2143?utm_source=github">Review
in cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
thanks @CalebBarnes for the contribution! (original PR: #2074)
# why
When running Stagehand locally, chrome-launcher adds its own set of
default flags (e.g. `--disable-extensions`). There's currently no way
for consumers to selectively remove these flags —
`localBrowserLaunchOptions.chromeFlags` can only *add* flags, not remove
chrome-launcher's built-in defaults.
This is needed when users want to run Chrome with extensions enabled, or
need to remove other chrome-launcher defaults for their use case.
# what changed
- Added `ignoreDefaultArgs` option to `LaunchLocalOptions` in `local.ts`
- Forwarded `localBrowserLaunchOptions.ignoreDefaultArgs` from `v3.ts`
through to `launchLocalChrome()`
- When `true`: drops all chrome-launcher defaults (only Stagehand's own
flags and user-supplied `chromeFlags` are used)
- When `string[]`: selectively removes only the listed flags from
chrome-launcher defaults, keeping the rest
This mirrors the behavior of Playwright's
[`ignoreDefaultArgs`](https://playwright.dev/docs/api/class-browsertype#browser-type-launch-option-ignore-default-args)
option.
# test plan
- Verified locally that setting `ignoreDefaultArgs:
["--disable-extensions"]` successfully removes the flag from the
launched Chrome instance (confirmed via `chrome://version/`)
- Verified that other chrome-launcher defaults are preserved when using
the array form
- Verified that `ignoreDefaultArgs: true` drops all chrome-launcher
defaults
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Adds `ignoreDefaultArgs` to local Chrome launch so you can drop all or
selected default flags from `chrome-launcher` with exact-match control
(including Stagehand defaults), while keeping your own `args`.
- **New Features**
- Added `ignoreDefaultArgs` to `LaunchLocalOptions` and moved flag
assembly into `launchLocalChrome()`, forwarding to `chrome-launcher` via
`ignoreDefaultFlags`.
- `true` drops all `chrome-launcher` defaults; `string[]` removes only
exact-matched defaults and re-adds the rest from
`Launcher.defaultFlags()`. Stagehand defaults and user `args` stay
unless explicitly listed.
<sup>Written for commit e04da99099ed354d775c1a67253358abdfd36fe3.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/2127?utm_source=github">Review
in cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
---------
---------
Co-authored-by: Caleb Barnes <caleb.d.barnes@gmail.com>
Co-authored-by: Caleb Barnes <CalebBarnes@users.noreply.github.com>
Co-authored-by: Mastra Code (anthropic/claude-opus-4-6) <noreply@mastra.ai>
thanks @asim48-ctrl for the contribution!
## Summary
- replace the remaining `ai/dist` deep import with the public `ai`
package export
- aligns `AgentProvider` with the other Stagehand v3 CUA files that
already import `ToolSet` from `ai`
Fixes#1531.
## Verification
- `corepack pnpm install --ignore-scripts --frozen-lockfile`
- `corepack pnpm --filter @browserbasehq/stagehand run gen-version`
- `corepack pnpm --filter @browserbasehq/stagehand run
build-dom-scripts`
- `corepack pnpm exec tsc -p packages/core/tsconfig.json --noEmit`
- `corepack pnpm exec prettier --check
packages/core/lib/v3/agent/AgentProvider.ts`
- `git diff --check`
<!-- This is an auto-generated description by cubic. --> ---
## Summary by cubic
Switch AgentProvider to import ToolSet from the public `ai` export
instead of `ai/dist` to prevent deep import breakage. Adds a patch
changeset and aligns with other Stagehand v3 CUA files; fixes#1531.
<sup>Written for commit 1c1767582d16ac2333f171ac87dc877dd6d6ba93.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
Co-authored-by: asim48-ctrl <asim48@gmail.com>
# why
- these were unneeded dependencies:
- they were not used in library code at all
- nor were they included in the public export surface
# what changed
- removed `lanchain/core` and `langchain/openai` from dependencies
- removed langchain examples
- updated lockfile
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Removed unused LangChain dependencies and examples from `packages/core`
to reduce install size and simplify the codebase. No API or runtime
behavior changes.
- **Dependencies**
- Removed `@langchain/core` and `@langchain/openai` from
`packages/core/package.json`.
- Updated `pnpm-lock.yaml` to drop related transitive packages.
- **Refactors**
- Deleted LangChain examples:
`packages/core/examples/langchain-client.ts` and
`packages/core/examples/custom-client-langchain.ts`.
<sup>Written for commit e46cfa4d1f946adac98769253cc2c01378bf9cd1.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
# why
- unused peer dep
# what changed
- removed `deepmerge` from peer deps
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Removed unused peer dependency `deepmerge` from `packages/core`. This
simplifies installs and removes unnecessary peer warnings.
<sup>Written for commit 50652b85227a06627d557c1c431b33c96470555c.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
# why
- users don't need to be installing our tests
# what changed
- added explicit negations for `dist/esm/tests` and `dist/cjs/tests` in
`packages/core/package.json`
- this keeps our test suite running with the built code in `dist`, while
still avoiding publishing test files to npm
# test plan
- verified that test are not included in tarball with `npm pack
--dry-run`
- existing test suite
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Stop publishing test files with the package to reduce the npm tarball
and avoid installing unnecessary files. Updated
`packages/core/package.json` to exclude `dist/esm/tests` and
`dist/cjs/tests` via the `files` array; tests still run against built
code locally and were verified with `npm pack --dry-run`.
<sup>Written for commit ccd7212304bcec1b8a8e32d32c035eb4263cb81e.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
thanks to @imorhun for this contribution! original PR can be found here
#2105
# why
Stagehand accessibility snapshot output was missing native selection
state markers (`selected` / `checked`) that are visible in Playwright
`ariaSnapshot({ mode: "ai" })`.
That mismatch made it impossible to extract the selected/checked item
for dropdowns and radio groups.
Example:
```html
<select aria-label="Select field">
<option>Option A</option>
<option selected>Option B</option>
<option>Option C</option>
</select>
```
Before this fix, Stagehand-style snapshot output did not indicate the
selected option:
```
[1-4] select: Select field
[1-5] option: Option A
[1-6] option: Option B
[1-7] option: Option C
```
Expected (and now aligned more closely with Playwright-style output):
selected state is explicit:
```
[1-4] select: Select field
[1-5] option: Option A
[1-6] option: Option B [selected]
[1-7] option: Option C
```
# what changed
- added explicit boolean fields on `A11yNode`:
- `selected?: boolean`
- `checked?: boolean`
- updated AX mapping in `a11yTree` to:
- read native AX `selected` / `checked` from AX properties
- normalize boolean-like AX values (`true/false`, `1/0`,
`"true"/"false"`)
- store the result directly on `selected` / `checked`
- updated tree formatting in `treeFormatUtils` to render:
- `[selected]` when `selected === true`
- `[checked]` when `checked === true`
# test plan
- AX selected/checked extraction into boolean fields
- select/radio parent-child rendering with exactly one selected/checked
child
- rendering both flags when a node carries both states
---------
Co-authored-by: Ihor <vreysu@gmail.com>
# why
- bumps to latest mintlify version with various transitive dependency
fixes
# what changed
- bumped mintlify to version `5.2.563`
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Bump `mintlify` in `packages/docs` to ^4.2.563 to pull in upstream fixes
and refresh transitive dependencies for the docs build.
- **Dependencies**
- Updated lockfile; notable bumps include `@mintlify/cli@4.0.1166`,
`@mintlify/common@1.0.895`, `@mintlify/previewing@4.0.1100`,
`@mintlify/validation@0.1.701`, `js-yaml@4.1.1`.
- Modernized tar stack (`tar@7`, `minizlib@3`, `chownr@3`, `yallist@5`),
aligning with Node 18+.
<sup>Written for commit 9bd8ff99768a8b4dce134163162535599892a4c6.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
# why
this updates a handful of deps we were behind on & avoids breaking docs
# what changed
bumped the following deps:
- `@langchain/openai` to `^1.4.5`
- `@langchain/core` to `^1.1.46`
- `@anthropic-ai/claude-agent-sdk` to `^0.2.141`
- `@fastify/swagger-ui` to `^5.2.6` in `packages/server-v3` &
`packages/server-v4`
- `uuid` to `^11.1.1`
- `vite` to `^7.3.3` in the packages that use `vitest`
- workspace tooling like `@changesets/cli`, `c8`, `eslint`, and `tsup`
added the following `pnpm` overrides:
- `axios`
- `follow-redirects`
- `jws`
- `lodash`
- `minimatch`
- `uuid`
also fixed a `vitest` mock typing issue in
`packages/evals/tests/tui/commandTree.test.ts` that resulted from the
vite bump
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Update workspace dependencies and add `pnpm` overrides to lock
transitive versions, stabilizing builds and preventing docs from
breaking. Also fixes a Vitest typing issue introduced by the Vite
upgrade.
- **Dependencies**
- Bumped: `@langchain/openai` → `^1.4.5`, `@langchain/core` → `^1.1.46`,
`@anthropic-ai/claude-agent-sdk` → `^0.2.141`, `uuid` → `^11.1.1`,
`@fastify/swagger-ui` → `^5.2.6` (server v3/v4).
- Tooling: `vite` → `^7.3.3`, `vitest` → `^4.1.6`, `tsup` → `^8.5.1`,
plus `@changesets/cli`, `c8`, `eslint`.
- Docs: `mintlify` → `^4.2.546` and move OpenAPI path to `api.openapi`
in `packages/docs/docs.json`.
- Added `pnpm.overrides` for `axios`, `follow-redirects`, `jws`,
`lodash`, `minimatch`, `uuid`.
- **Bug Fixes**
- Fixed `vitest` mock typing in
`packages/evals/tests/tui/commandTree.test.ts` by typing handlers
(`CommandHandler`) and `printHelp` mocks.
<sup>Written for commit cd57bdf55f36ffe05bfa0de3e43aa2619d159546.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
# why
# what changed
# test plan
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Revamped the `packages/evals` README to explain the current Stagehand
Evals TUI/CLI: quickstart, commands, run targets, common flags
(harness/agent modes), preview/progress/results, task scaffolding, and
Braintrust tracing. Added screenshots plus a run demo GIF, clarified
`.env` loading, and removed outdated notes to match auto-discovered
tasks and the `evals` binary.
<sup>Written for commit 0f80088278a6d0c37285ee75a2f9f67973d61814.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
# why
Improving the evals onboarding experience
# what changed
Added a Welcome screen on first run after install on TUI and a net new
`evals doctor` subcommand to triage common issues with
installation/setup
<img width="732" height="814" alt="Screenshot 2026-05-11 at 12 08 41 PM"
src="https://github.com/user-attachments/assets/c7ae377b-3219-4b42-b8c6-e70182b76709"
/>
# test plan
- [x] 500+ LOC changes of new tests
- [x] Existing test suite passess
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Adds a first-run onboarding flow to the `evals` REPL and a new `evals
doctor` (alias `health`) command for environment and setup diagnostics.
Integrates with the command-tree dispatcher, adds a REPL-only quiet
mode, and persists a `_meta` first-run marker so the welcome shows once.
- **New Features**
- REPL onboarding: one-time welcome; suppress with `--quiet`/`-q`
(REPL-only) or `EVALS_NO_WELCOME=1`. Banner shows art only. When no
welcome, print a single inline warning only if no provider keys;
otherwise a compact tip.
- First-run state: writes `_meta.firstRunCompletedAt` (with version) to
`evals.config.json`; preserved across CLI rebuilds. REPL marks
pre-prompt unless `--quiet`. Argv marks only after real handlers run.
Help/meta paths and `doctor` (including nested help) do not mark.
- `evals doctor`/`health`: human output and `--json` (always exits 0);
verdicts `ok|warn|fail` with exit codes `0|0|1`. Hidden `--probe`
verifies an OpenAI key. Report includes runtime (Node, Stagehand, mode),
config (path/defaults/core), task discovery, and a key matrix
(OpenAI/Anthropic/Google/Browserbase/Braintrust) with source provenance
and Browserbase alias hints. Listed in help and callable from the REPL.
- **Migration**
- Run `evals doctor` to verify setup. Set
`OPENAI_API_KEY`/`ANTHROPIC_API_KEY`/`GOOGLE_GENERATIVE_AI_API_KEY` (or
`GEMINI_API_KEY`) and `BRAINTRUST_API_KEY`; for `env=browserbase`, set
`BROWSERBASE_API_KEY` and `BROWSERBASE_PROJECT_ID` (or `BB_*`).
- To skip onboarding on launch, use `--quiet` or set
`EVALS_NO_WELCOME=1`.
<sup>Written for commit ed61696358943a09540249b947cf287e45f3e698.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
# Why
`AnthropicCUAClient.convertToolUseToAction()` handled actions like
`click`, `double_click`, and `left_click`, but did not explicitly
support `triple_click`.
Two issues:
1. The action type remained in snake_case (`triple_click`) instead of
being normalized to `tripleClick` — the format expected by
`v3CuaAgentHandler.executeAction()`.
2. Coordinates provided via Anthropic’s coordinate: [x, y] format were
not converted into the separate x / y fields required by the handler.
As a result, Anthropic triple-click actions were silently ignored by the
handler switch.
# What Changed
- AnthropicCUAClient.ts
- Added explicit handling for triple_click / tripleClick in
convertToolUseToAction()
- Mirrors the existing double_click implementation
- Normalizes action type to tripleClick
- Extracts coordinates from both:
- coordinate: [x, y]
- direct x / y fields
- v3CuaAgentHandler.ts
- Added "triple_click" as a switch-case alias in executeAction()
# Test Plan
Added `anthropic-cua-triple-click.test.ts` (Vitest) covering:
- triple_click with coordinate: [x, y]
- triple_click with direct x / y fields
All existing unit tests continue to pass.
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes Anthropic CUA `triple_click` so it runs as `tripleClick` with x/y
parsed from `coordinate: [x, y]` or explicit fields. Also adds a CI job
to run evals unit tests.
- **Bug Fixes**
- Normalize `triple_click`/`tripleClick` to `tripleClick` in
`AnthropicCUAClient.convertToolUseToAction()`.
- Parse `x`/`y` from `coordinate: [x, y]` or direct fields.
- Add `"triple_click"` alias in `v3CuaAgentHandler.executeAction()`.
- **Dependencies**
- Changeset to publish a patch for `@browserbasehq/stagehand`.
<sup>Written for commit a230bfcd987b1ec229956f338ac8caa4b58c0a2f.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
---------
Co-authored-by: Chromie <miguel@browserbase.com>
Co-authored-by: Claude <noreply@anthropic.com>
# what changed
- adds new param to reference section
- adds example to "targeted extract" section
<img width="723" height="974" alt="Screenshot 2026-05-06 at 10 36 25 AM"
src="https://github.com/user-attachments/assets/35d676c6-48f0-46c7-8b76-cf319fa7af5e"
/>
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Documented the new `ignoreSelectors` option for `extract()` to exclude
elements (and their descendants) from the snapshot. Added a
targeted-extract example and clarified how it interacts with `selector`,
plus minor wording updates for supported page types.
<sup>Written for commit f039e4503888986e418a3fb4688e11af6551698b.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
# why
- to add docs for new `ignoreSelectors` param in `observe()`
# what changed
- adds new param to reference section
- adds an example snippet in 'the basics' section
<img width="761" height="792" alt="Screenshot 2026-05-07 at 11 41 26 AM"
src="https://github.com/user-attachments/assets/8e6c636a-a9ed-4c56-ad28-9030c532d995"
/>
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Added docs for the new `ignoreSelectors` option in `observe()` to let
users exclude page regions from observation. Updated the reference with
type and behavior, plus a basics example using XPath and a note on how
it interacts with `selector`.
<sup>Written for commit 35f9577986ff7ec134fbe8c3927f605d39128eb6.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
# why
# what changed
# test plan
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Unifies the Evals TUI/CLI behind a shared command tree with path-style
traversal and contextual help. Supports Linear STG-1880 and fixes root
fallback parsing by stripping a leading `evals` sigil to improve
onboarding and command discovery.
- **New Features**
- Single `buildCommandTree` powers both REPL and argv dispatch.
- `>` works as a path separator in REPL and argv; REPL preserves quoted
`>`, argv re-splits any arg containing `>`.
- Context navigation and prompt: `..` goes up; bare `evals` jumps to
root; invoking a node with children auto-enters that context in REPL;
prompt shows the current path; banner mentions `..`.
- Help routing: `--help` after a leaf calls that node’s help; `help`
shows help for the current context.
- Root fallback: unknown tokens at root call `run` and now strip a
leading `evals` before forwarding (e.g., `evals act`); unknown at depth
errors for clarity.
- REPL quality of life: Esc pops context when idle; Esc still aborts
in-flight runs (cooperative/double-press aggressive).
- **Refactors**
- `cli.ts` and `repl.ts` now resolve and execute via `dispatch` from the
shared command tree; added `tokenizeArgv` for argv `>` splitting and
`renderPrompt` for context-aware prompts.
- Moved line tokenization to `tui/tokenize.ts`; added
`tui/commandTree.ts` with resolution, prompt rendering, and argv
tokenization.
- Kept command handlers lazy-loaded; added focused tests for the tree,
`>` chaining, argv sigil handling, and CLI edge cases.
<sup>Written for commit 2f5c929685407e78762a0938b58f989e831aab6d.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
---------
Co-authored-by: Chromie <miguel@browserbase.com>
Co-authored-by: Claude <noreply@anthropic.com>
# why
- changsets are not properly set up in evals workspace, & this package
is not yet published on npm
# what changed
- removed patch changeset for evals pkg
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Removed the patch changeset for `@browserbasehq/stagehand-evals` to
prevent an accidental release. The evals workspace isn’t configured for
Changesets and the package isn’t published to npm.
<sup>Written for commit 86a520040ad50a933ea18d2defe0a2f8c17c9ba9.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
# why
- pnpm installs optional deps by default, which meant installing
`stagehand` would also pull in `playwright-core`, `patchright-core`, &
`puppeteer-core`
- these packages should be supplied by the consumer **_only if_** they
want to use `stagehand` with an existing browser automation framework.
they are otherwise not required
- moving them into peer dependencies prevents unnecessary installs, &
reflects a more accurate dependency graph
# what changed
- moved `playwright-core`, `patchright-core`, & `puppeteer-core` in
`packages/core` from optional deps to optional peer deps
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Moved `playwright-core`, `patchright-core`, and `puppeteer-core` to
optional peer dependencies in `@browserbasehq/stagehand` to stop
automatic installs and let consumers opt in only if needed. Also aligned
`playwright` version ranges across packages.
- **Dependencies**
- In `packages/core`: moved `playwright-core`, `patchright-core`,
`puppeteer-core` from optional deps to optional peer deps (all marked
optional).
- Updated internal dev deps to keep tests working: `playwright`,
`playwright-core`, `puppeteer-core`, `patchright-core`.
- Aligned `playwright` ranges: `packages/evals` uses `>=1.55.1 <1.57.0`;
`packages/server-v3` and `packages/server-v4` use `^1.55.1`.
- **Migration**
- If you use Stagehand with Playwright, add `playwright-core@^1.55.1`.
- If you use Puppeteer, add `puppeteer-core@^24.43.0`.
- If you use Patchright, add `patchright-core@^1.55.2`.
<sup>Written for commit dc726fc12933305106cb7936eb6a18310cfb7c53.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
# why
- lockfile was unused, leftover from monorepo reorg
# what changed
- removed `packages/docs/pnpm-lock.yaml`
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Removed the unused docs lockfile left over from the monorepo reorg.
Deleting `packages/docs/pnpm-lock.yaml` reduces noise and avoids
confusion about dependency state.
<sup>Written for commit d8a5c2b1da751e3edf2c2271382bb7984278e364.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
# Why
Help discovery was inconsistent across the evals CLI.
- `--help` / `-h` worked everywhere.
- The bare word `help` only worked at the top level and on three
dispatcher commands.
- `config core` had no dedicated help page at all — `evals config core
--help` was intercepted upstream and printed the parent `config` help
instead.
# What Changed
- Added `printConfigCoreHelp` documenting:
- `evals config core`
- `path`
- `set`
- `reset`
- `setup`
- Standardized `help` as a positional alias for `--help` at every level:
- Top-level dispatchers:
- `run`
- `list`
- `new`
- `config`
- `experiments`
- Nested dispatchers:
- `config core`
- Leaf verbs:
- `set`
- `reset`
- `path`
- `setup`
- `list`
- `show`
- `open`
- `compare`
- Removed the early `wantsHelp → printConfigHelp()` interception in:
- `cli.ts`
- `repl.ts`
This allows `config core --help` to correctly reach the core-specific
help page.
- Cleaned up:
- unused imports
- dead experiments help branch
- Help interception now only occurs when `help` appears as the first
argument to a verb, so legitimate values such as `evals config set
provider help` continue to work properly.
# Tests
- [x] Added 35 parameterized `it.each` cases in `tests/cli.test.ts`
covering:
- `help`
- `--help`
- `-h`
across every command and subcommand level.
- [x] Full evals unit suite:
- `251 / 251` passing
- [x] Verified built CLI prints correct help for:
- `config core help`
- `experiments compare help`
- `run help`
# why
- to bump a small set of direct and transitive dependencies to pick up
patched upstream releases
# what changed
- bumped `playwright` to 1.55.1 across workspace all workspace packages
- bumped `@playwright/test` in `packages/core` to 1.55.1
- bumped `playwright-core` in `packages/core` to 1.55.1
- bumped `puppeteer-core` in `packages/core` to 24.43.0
- bumped `jsdom` in `packages/core` to 29.1.1.
- bumped `@browserbasehq/sdk` where needed to pick up a patched
transitive dependency tree
# test plan
- existing test suite + pnpm audit
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Bumps key direct and transitive dependencies across the monorepo to pick
up patched upstream releases and align browser tooling. Ships patch
releases for Stagehand packages.
- **Dependencies**
- `packages/core`: `playwright` → `^1.59.1`, `playwright-core` →
`^1.59.1`, `@playwright/test` → `^1.59.1`, `puppeteer-core` →
`^24.43.0`, `jsdom` → `^29.1.1`, `openai` → `^4.104.0`,
`@langchain/openai` → `^0.4.9`.
- `packages/server-v3` and `packages/server-v4`: `playwright` →
`1.55.1`.
- `packages/evals`: `openai` → `^4.104.0`, `playwright` → `^1.55.1`,
`braintrust` → `^0.4.10`, added `@types/node` and `types: ["node"]` in
`tsconfig`.
- Updated `pnpm-lock.yaml`.
- Patch releases: `@browserbasehq/stagehand`,
`@browserbasehq/stagehand-evals`, `@browserbasehq/stagehand-server-v3`,
`@browserbasehq/stagehand-server-v4`.
- **Migration**
- Reinstall deps: `pnpm install`.
- Reinstall Playwright browsers if needed: `npx playwright install`.
<sup>Written for commit db96e496ead3c081d74484784763aa86b6d096a9.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
## Summary
- default the v3 server listener to localhost unless HOST is explicitly
set
- warn when HOST=0.0.0.0 opts into all-interface binding
- make the v4 SEA build choose a Node binary that contains the SEA fuse
## Validation
- pnpm --filter @browserbasehq/stagehand-server-v3 run build:esm-tests
- pnpm --filter @browserbasehq/stagehand-server-v3 run typecheck
- pnpm --filter @browserbasehq/stagehand-server-v4 run typecheck
- pnpm --filter @browserbasehq/stagehand-server-v3 run test:unit --
packages/server-v3/dist/tests/unit/listenHost.test.js
- pnpm --filter @browserbasehq/stagehand-server-v4 run build:sea:esm --
--binary-name=stagehand-server-v4-localhost-binding-test
Leaving the full suite to CI.
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Default `@browserbasehq/stagehand-server-v3` to listen on localhost
unless `HOST` is set, and warn on `HOST=0.0.0.0` to prevent accidental
exposure. Also make the `@browserbasehq/stagehand-server-v4` SEA build
use a Node binary that includes the SEA fuse for reliable builds.
- New Features
- Default listen host to `localhost` when `HOST` is unset or blank;
respect explicit values.
- Warn when `HOST=0.0.0.0` to make all-interface binding explicit.
- Migration
- If you need external access, set `HOST=0.0.0.0` (or a specific
interface).
<sup>Written for commit 3cecdce8251f43aa3853ca56f8924eb0dda9dff7.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
# why
- to let users exclude parts of the page from the snapshot that
`observe()` sends to the model, the same way they can with `extract()`
# what changed
- this PR is just wiring the added param through observe & updating
types. it uses the same underlying functionality that was added in #2084
when adding `ignoreSelectors` to `extract()`
- `packages/core/lib/v3/types/public/methods.ts`,
`packages/core/lib/v3/types/private/handlers.ts`, &
`packages/core/lib/v3/types/public/api.ts`:
- added the `ignoreSelectors` option to the public api, internal handler
params, & request schema
- `packages/core/lib/v3/v3.ts` &
`packages/core/lib/v3/handlers/observeHandler.ts`:
- passed `ignoreSelectors` through the observe path into
`captureHybridSnapshot()`
# test plan
- updated `packages/core/tests/unit/public-api/public-types.test.ts` to
cover the new `ignoreSelectors` param
- updated `packages/core/tests/unit/api-variables-schema.test.ts` &
`packages/core/tests/unit/api-client-observe-variables.test.ts` to cover
request validation and serialization for `observe({ ignoreSelectors })`
- updated `packages/core/tests/unit/timeout-handlers.test.ts` to cover
`ObserveHandler` passing `ignoreSelectors` into
`captureHybridSnapshot()`
### notes
- the test file placement is a little weird right now:
- `timeout-handlers.test.ts` already contains broader parameter
threading tests, so this was the smallest change for now
- this can be cleaned up in a later refactor
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Add `ignoreSelectors` to `observe()` so you can exclude elements from
the snapshot sent to the model. Brings `observe()` to parity with
`extract()`.
- **New Features**
- Support `observe({ ignoreSelectors: ["nav", ".cookie-banner"] })` to
skip elements and subtrees during snapshot capture.
- Plumbed through public types, internal handler params, and
`captureHybridSnapshot()` call path.
- Request schema and `packages/server-v3/openapi.v3.yaml` updated;
client serialization and validation covered by tests.
- Published as a minor for `@browserbasehq/stagehand`.
<sup>Written for commit c7b538c04d439b13d28db843a12344529171219b.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
## Summary
- remove ModelApiKey from the server-v3 OpenAPI global security
requirement
- update the checked-in v3 OpenAPI spec so Stainless no longer treats
model API key as required client auth
## Verification
- pnpm --filter @browserbasehq/stagehand-server-v3 run lint
- pnpm --filter @browserbasehq/stagehand-server-v3 run build
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Make the model API key optional by removing `ModelApiKey` from server v3
OpenAPI global security (STG-1806). Update the spec/generator in
`@browserbasehq/stagehand-server-v3` and add a patch changeset so
Stainless clients no longer require a model key.
- **Migration**
- Regenerate clients from the v3 OpenAPI (e.g., Stainless) to drop
`ModelApiKey` as required auth.
<sup>Written for commit 3c74893febe6a3501f26eaa700ba2290c02cad56.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
---------
Co-authored-by: samfinton <samfinton@samLaptopm5.galaxus.box>
# why
- schema descriptions changed which means spec yaml files need to be
regenerated
- the regenerated yaml files were uncommitted
# what changed
- added the regenerated yaml files
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Regenerated OpenAPI specs for `packages/server-v3` and
`packages/server-v4` to align with schema description updates. In v4,
the `x-bb-project-id` header and related `projectId` fields are now
deprecated since API keys are project-scoped.
- **Migration**
- Stop sending the `x-bb-project-id` header.
- Omit `projectId` in v4 session payloads and ignore it in responses.
<sup>Written for commit 8dd382e1833337e8ea52021f8ca66f9284e01061.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
Linear:
https://linear.app/browserbase/issue/STG-1935/strengthen-observe-element-id-prompting
## Summary
- keep the strict `number-number` observe element ID schema intact
- strengthen the observe system prompt to tell the model to copy the
full bracketed tree ID, including frame ordinal and hyphen
- strengthen the schema description with the concrete `[0-18372] ->
"0-18372", not "18372"` example
- add prompt regression coverage, main-frame `0-...` observe integration
coverage, and a patch changeset
- add `observe/observe_main_frame_element_ids` as a regression eval for
multiple main-frame targets
## Eval Coverage
Current bench eval count after this PR:
```text
146 total bench evals
12 observe bench evals
```
This PR adds one observe regression eval:
`observe/observe_main_frame_element_ids`.
## Validation
- `pnpm --filter @browserbasehq/stagehand run build`
- `pnpm --filter @browserbasehq/stagehand run test:core --
packages/core/dist/esm/tests/unit/prompt-observe-variables.test.js`
- `pnpm --filter @browserbasehq/stagehand run test:e2e --
packages/core/dist/esm/tests/integration/observe-element-id-format.spec.js`
- `pnpm --filter @browserbasehq/stagehand run lint`
- `pnpm --filter @browserbasehq/stagehand-evals run typecheck`
- `pnpm --filter @browserbasehq/stagehand-evals run test:unit --
tests/taskConfig.test.ts`
- `pnpm --filter @browserbasehq/stagehand-evals run lint`
- `env STAGEHAND_E2E_TARGET=BROWSERBASE pnpm exec tsx
/Users/shrey/Developer/scratchpad/stagehand-observe-bare-id-e2e/index.ts`
- `pnpm --filter @browserbasehq/stagehand-evals exec tsx cli.ts run
observe_main_frame_element_ids -e browserbase -t 1 -c 1 -m
anthropic/claude-sonnet-4-6`
Real Browserbase E2E used `anthropic/claude-sonnet-4-6` and verified raw
model IDs plus click success:
```text
Attempt 1 raw model elementId: 0-5011
Attempt 1 act success: true
Attempt 1 DOM clicked marker: yes
Attempt 2 raw model elementId: 0-5011
Attempt 2 act success: true
Attempt 2 DOM clicked marker: yes
Attempt 3 raw model elementId: 0-5011
Attempt 3 act success: true
Attempt 3 DOM clicked marker: yes
```
The new Browserbase bench eval also passed with
`anthropic/claude-sonnet-4-6`:
```text
observe/observe_main_frame_element_ids anthropic/claude-sonnet-4-6 passed
Results: 1 passed, 0 failed (1 total)
Pass rate: 100%
```
# why
Stagehand v3 was setting hard-coded temperature defaults for standard
act, extract, and observe inference calls. This forced provider
behavior instead of letting model/provider defaults apply.
# what changed
Removed the hard-coded temperature: isGPT5 ? 1 : 0.1 defaults from
shared act, extract, and observe inference.
Removed the unused GPT-5 temperature branching in that path.
Updated Groq and Cerebras clients to pass through options.temperature
directly instead of defaulting missing values to 0.7, while preserving
explicit temperature: 0.
Added regression tests covering:
- act, extract, and observe no longer pass temperature by default
- Groq/Cerebras do not inject default temperature
- Groq/Cerebras preserve explicit zero temperature
# test plan
- pnpm --filter @browserbasehq/stagehand run typecheck
- pnpm --filter @browserbasehq/stagehand exec prettier --check
lib/inference.ts lib/v3/llm/GroqClient.ts lib/v3/llm/CerebrasClient.ts
tests/
unit/inference-temperature.test.ts
tests/unit/openai-compatible-temperature.test.ts
- pnpm --filter @browserbasehq/stagehand run build:esm
- pnpm --filter @browserbasehq/stagehand run test:core --
packages/core/dist/esm/tests/unit/inference-temperature.test.js
packages/core/
dist/esm/tests/unit/openai-compatible-temperature.test.js
packages/core/dist/esm/tests/unit/aisdk-clients.test.js
packages/core/dist/esm/
tests/unit/anthropic-cua-adaptive-thinking.test.js
---------
Co-authored-by: Filip Michalsky <filip-michalsky@users.noreply.github.com>
Co-authored-by: Miguel <36487034+miguelg719@users.noreply.github.com>
Co-authored-by: Chromie <miguel@browserbase.com>
Co-authored-by: Claude <noreply@anthropic.com>
# why
`experimental: true` is no longer required for agent variable support
# what changed
- Removed the experimental gate on agent variables: dropped the
variables check in `validateExperimentalFeatures.ts` and the
@experimental JSDoc on `AgentExecuteOptionsBase.variables`.
- Added a variables field to the AgentExecuteOptions API schema
(`packages/core/lib/v3/types/public/api.ts`) and reflected it in the
generated `openapi.v3.yaml` (both request and output components).
- Updated docs to drop `experimental: true` from the Variables Note and
Basic Usage example (kept the "Non-CUA agents only" caveat).
- Added unit tests covering the new surface
- Added integration coverage on the login eval site: new variables
describe blocks in `act.test.ts` and `agentExecute.test.ts` (simple and
rich { value, description } forms, verified via CDP), and updated
`observe.test.ts`'s existing variables tests to use the same scenario.
# test plan
- [x] pnpm --filter @browserbasehq/stagehand-server-v3 typecheck —
passes.
- [x] pnpm exec prettier --check on the three test files — passes.
- [x] pnpm exec eslint on the three test files — passes.
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Enable variables in v3 `agentExecute` without experimental mode, with
end-to-end substitution across `act`, `observe`, and agent tools.
Tightened `fillFormVision` so secrets are only substituted at type-time
and never leak via logs, replays, or tool outputs.
- **New Features**
- Added `variables` to `AgentExecuteOptionsSchema` and server OpenAPI
(`Variables`).
- Removed experimental gating for variables; CUA-mode rejection remains.
- Updated JSDoc and docs to drop the experimental requirement.
- Added tests: unit (validation, client serialization, schema) and
integration for substitution in `act`, `observe`, and `agentExecute`
(simple and rich forms).
- **Bug Fixes**
- Updated `fillFormVision` to substitute variables only when typing and
keep placeholders in action logs, replays, and returned arguments.
<sup>Written for commit a6e9d621d396cab397bd5348c814629373693cf0.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
---------
Co-authored-by: Chromie <miguel@browserbase.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: tkattkat <tkat@tkat.net>
# why
Browserbase API keys are project-scoped, so Stagehand should keep
accepting project IDs without requiring them.
# what changed
- Mark `browserbaseSessionCreateParams.projectId` deprecated in
Zod/OpenAPI.
- Make `BROWSERBASE_PROJECT_ID` nullable/deprecated in Stainless config.
- Use Browserbase-returned `projectId` in the local server instead of
`x-bb-project-id`.
```ts
projectId: z.string().optional().meta({ deprecated: true })
```
# test plan
- `pnpm --filter @browserbasehq/stagehand run lint`
- `pnpm --filter @browserbasehq/stagehand-server-v3 run lint`
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Deprecate Browserbase project ID per STG-1808; Stagehand now relies on
project-scoped API keys and the project ID returned by Browserbase. The
`x-bb-project-id` header, session `projectId` param, and
`BROWSERBASE_PROJECT_ID` config are deprecated but remain optional for
now.
- **Refactors**
- Marked `projectId` in session create params as deprecated in
Zod/OpenAPI; updated security scheme descriptions to reflect deprecation
of `x-bb-project-id`.
- Made `BROWSERBASE_PROJECT_ID` nullable and marked deprecated in
`stainless.yml`.
- Server now resolves `projectId` from the retrieved/created Browserbase
session; stops reading `x-bb-project-id` and adjusts validation
messages.
- **Migration**
- Stop sending `x-bb-project-id` and remove `BROWSERBASE_PROJECT_ID`;
they’re no longer needed.
- Avoid passing `projectId` in session creation; it’s accepted but
deprecated.
<sup>Written for commit 5ee90b860105739405927eec695060a1d1a066fc.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/2039">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
# why
# what changed
# test plan
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Adds a manual GitHub Actions workflow to run evaluation suites on demand
with selectable model, agent mode, trials, and concurrency. Builds once,
runs chosen evals in a matrix, and posts Braintrust results and
summaries.
- **New Features**
- Run agent, act/extract/observe, or external agent benchmarks
(webvoyager, onlineMind2Web, webtailbench).
- Inputs: `model`, `agent_mode` (auto/dom/hybrid/cua), `trials` (default
1), `concurrency` (default 50).
- One build job; artifacts reused across eval jobs. Dynamic matrix built
via `jq`.
- Executes evals with `pnpm` + `turbo` against
`@browserbasehq/stagehand-evals`, with
OpenAI/Anthropic/Google/Braintrust/Browserbase env wired and region
selection.
- Publishes job summary with Braintrust link and primary score, and
uploads CTRF report + V8 coverage (fails if no summary).
<sup>Written for commit 8267815433cfbbac5988d1f6b827be62ea43bade.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
# why
- to allow users to specify selectors whose content should be omitted
from the LLM request inside of `extract()`
# what changed
- added `ignoreSelectors` to `extract()` so callers can exclude parts of
the page from the extraction snapshot
- `packages/core/lib/v3/understudy/a11y/snapshot/capture.ts`:
- added `resolveIgnoreSelectorRoots()` to resolve ignore selectors into
`frameId + backendNodeId`
- added `buildFrameExclusionIntervals()` to turn those roots into
per-frame subtree ranges
- updated `collectPerFrameMaps()` / `tryScopedSnapshot()` to skip
ignored nodes when building snapshot data
- `packages/core/lib/v3/understudy/a11y/snapshot/domTree.ts`:
- updated `buildSessionDomIndex()` to record dfs entry/exit positions
for each backend node so ignored subtrees can be filtered by interval
membership instead of having to re-traverse the tree
- `packages/core/lib/v3/understudy/a11y/snapshot/a11yTree.ts`:
- updated `a11yForFrame()` to accept an ignore predicate and remove
ignored nodes from the outline (the stringified tree) and `urlMap`
- `packages/core/lib/v3/handlers/extractHandler.ts` and
`packages/core/lib/v3/v3.ts`:
- passed `ignoreSelectors` down through the extract path into
`captureHybridSnapshot()`
- kept `xpathMap` values based on the original dom structure and only
skipped ignored entries, so surviving selectors do not get renumbered
# test plan
- updated `packages/core/tests/unit/public-api/public-types.test.ts` to
cover the new `ignoreSelectors` option on `extract()`
- updated `packages/core/tests/unit/snapshot-a11y-resolvers.test.ts` to
cover ignored backend nodes being removed from the a11y outline and
`urlMap`, and to cover scoped snapshots dropping ignored xpath entries
- updated
`packages/core/tests/unit/snapshot-capture-orchestration.test.ts` to
cover the new capture flow, including filtered per-frame maps and merged
snapshot output when `ignoreSelectors` is used
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Adds `ignoreSelectors` to `extract()` so callers can exclude elements
and their subtrees from snapshots and the LLM input. Supports CSS and
XPath (with iframe hops) across shadow DOM and iframes, without
renumbering existing XPaths; also adds a fast path for scoped snapshots
and excludes child frame subtrees when an iframe host is ignored.
- **New Features**
- `extract({ ignoreSelectors: string[] })`: drops matching nodes and
descendants from DOM maps, a11y outline, and `urlMap`; `xpathMap` keeps
original values but omits ignored entries.
- Applies to full-page and scoped snapshots; filters resolved per frame,
excludes child frames when their iframe host is ignored, and enforces
DFS enter/exit intervals for speed.
- A11y tree accepts an ignore predicate and filters before building the
outline and links.
- Scoped snapshots return early when only `selector` is provided (no
`ignoreSelectors`) for faster capture.
- Plumbed through `extractHandler`, `v3`, and `captureHybridSnapshot()`;
public types and OpenAPI updated in `@browserbasehq/stagehand`; unit
tests added.
<sup>Written for commit 0ad2f67d8fe81d15945599c60351ac45f66f1277.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
# why
Reasoning models (e.g. `openai/gpt-5-mini`) emit `AI SDK Warning: The
"temperature" setting is not supported by this model` whenever the v3
agent
runs against them, because `V3AgentHandler` hardcoded `temperature: 1`
on every `generateText`/`streamText` call. There was no constructor or
agent-config option to override it, so users running reasoning models
with `experimental: true` had no way to silence the warning.
# what changed
- Removed the hardcoded `temperature: 1` from both the non-streaming and
streaming agent paths in
`packages/core/lib/v3/handlers/v3AgentHandler.ts`.
- Agent calls now omit `temperature` entirely, so the AI SDK no longer
flags it as user-set on models that don't accept it. Models that do
accept
temperature still default to the AI SDK provider's own default.
- Added unit coverage in
`packages/core/tests/unit/agent-temperature.test.ts` asserting neither
code path passes a `temperature` to the LLM
client.
- Added a patch changeset
(`.changeset/remove-agent-temperature-default.md`).
# test plan
- [ ] `pnpm --filter @browserbasehq/stagehand test agent-temperature` —
new unit tests pass
- [ ] Existing v3 agent unit suite stays green
- [ ] Manual: `agent.execute(...)` against `openai/gpt-5-mini` with
`experimental: true` — confirm no `"temperature" setting is not
supported`
warning in stderr
- [ ] Manual: same call against a non-reasoning model (e.g.
`openai/gpt-4.1-mini`) — confirm agent still completes a multi-step task
end-to-end
- [ ] Manual: hybrid-mode agent run on a reasoning model — verify
streaming path is also clean
---------
Co-authored-by: Filip Michalsky <filip-michalsky@users.noreply.github.com>
# why
Stagehand could lose a known parent-child frame relationship after
adopting an OOPIF child session. In practice, that meant the registry
could forget that a child frame still belonged under the top-level page,
leaving later iframe resolution with an incomplete frame tree.
# what changed
Added a deterministic FrameRegistry unit test for the OOPIF adoption
ordering. Updated `seedFromFrameTree()` to preserve an already-known
parent when a child frame tree snapshot does not include parent
metadata.
# test plan
Before the fix, the new test failed with `expected null to be
'root-frame'`. After the fix, this passes:
`pnpm --dir packages/core run build:esm && pnpm --dir packages/core run
test:core --
packages/core/dist/esm/tests/unit/frame-registry-oopif-adoption.test.js`
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes a bug where the frame registry could lose a known parent-child
link after adopting an OOPIF child session. Preserves the parent edge
when seeding from a child frame tree that lacks parent metadata to keep
the iframe tree complete.
- Bug Fixes
- Updated `FrameRegistry.seedFromFrameTree()` to retain an existing
parent when `parentId` is missing and to sync child sets accordingly.
- Added a unit test for OOPIF adoption ordering and seeding to prevent
regressions, and a changeset for `@browserbasehq/stagehand` patch
release.
<sup>Written for commit 1e50e9c3ed4d56c158c1b694bbd04d2aac55821c.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
---------
Co-authored-by: samfinton <samfinton@samLaptopm5.galaxus.box>
# why
Stagehand should standardize public docs and runtime overrides on
`STAGEHAND_API_URL` without breaking users who still export
`STAGEHAND_BASE_URL`.
# what changed
- Prefer `STAGEHAND_API_URL`, with `STAGEHAND_BASE_URL` as a deprecated
legacy fallback.
- Emit a one-time deprecation warning when the legacy fallback is used.
- Update server-v3 test env plumbing, CI, and docs to use
`STAGEHAND_API_URL`.
- Add patch changesets for core and server-v3.
```ts
const configuredBaseUrl =
process.env.STAGEHAND_API_URL ?? process.env.STAGEHAND_BASE_URL;
```
# test plan
- `pnpm --filter @browserbasehq/stagehand run build:esm`
- `pnpm --filter @browserbasehq/stagehand run test:core --
packages/core/dist/esm/tests/unit/api-optional-model-api-key.test.js`
- `pnpm --filter @browserbasehq/stagehand run typecheck`
- `pnpm --filter @browserbasehq/stagehand-server-v3 run typecheck`
- `pnpm --filter @browserbasehq/stagehand run lint`
- `pnpm --filter @browserbasehq/stagehand-server-v3 run lint`
- `pnpm exec prettier --check .changeset/rude-tips-check.md
.github/workflows/ci.yml packages/docs/v3/sdk/java.mdx
packages/docs/v3/sdk/python.mdx turbo.json`
- `pnpm changeset status --since origin/main --verbose`
# why
# what changed
# test plan
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Removed an accidental development transcript file from the repo to
reduce noise and avoid shipping non-code artifacts. No runtime or build
impact.
<sup>Written for commit 25c03785632e6ca0781b6d6dc8a2fbb58ea9daac.
Summary will update on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
# why
Dom mode has been deprecated for a while, and is now being replaced with
hybrid as a default
# what changed
- When a hybrid capable model is provided, it is used with hybrid mode
by default
- When a non hybrid capable model is provided, it is used with dom mode
- if dom or hybrid is explicitly provided, it will use the provided mode
# test plan
tested locally + wrote tests
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Set the default v3 agent mode to hybrid, with automatic routing to dom
for models that aren’t hybrid-capable. Explicit mode settings are always
respected; CUA behavior is unchanged.
- **New Features**
- Auto-selects "hybrid" for models matching
`HYBRID_CAPABLE_MODEL_PATTERNS` (`gemini-3`, `claude`, `gpt-5.4`,
`gpt-5.5`); otherwise uses "dom".
- Routes using the provided agent model (string or config), or falls
back to the Stagehand-level model when undefined.
- Honors explicit `mode` (`dom`, `hybrid`, `cua`); if `mode` is unset,
deprecated `cua: true` enables CUA.
- Logs the resolved mode and warns when "hybrid" is used with
non-hybrid-capable models.
- Adds unit tests for routing and handler fallbacks.
<sup>Written for commit 01e7d69945369cca735dff9ba088538420e344c9.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/2047">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
## What Changed
Adds three new models to the list of supported CUA (Computer Use Agent)
models:
- `openai/gpt-5.4-mini`
- `openai/gpt-5.5`
- `anthropic/claude-haiku-4-5`
---
## Files Changed
### `packages/core/lib/v3/types/public/agent.ts`
- Added the three models to the `AVAILABLE_CUA_MODELS` const array
- This drives:
- The `AvailableCuaModel` type
- The runtime CUA model check in `resolveModel()`
### `packages/core/lib/v3/agent/AgentProvider.ts`
- Added model → provider mappings in `modelToAgentProviderMap`:
- `gpt-5.4-mini` → `openai`
- `gpt-5.5` → `openai`
- `claude-haiku-4-5` → `anthropic`
- Ensures `AgentProvider.getAgentProvider()` resolves them correctly
### `packages/core/tests/unit/public-api/llm-and-agents.test.ts`
- Updated the `expectedModels` list in the type-level test to stay in
sync with the source of truth
---
## How It Was Tested
- ✅ Build passes (`pnpm turbo run build`)
- ✅ All **204 public-api unit tests** pass
- ✅ All **608 unit tests** pass
- ⚠️ 4 pre-existing failures in `flowlogger-eventstore.test.js`
(confirmed on base branch)
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Adds support for three new Computer Use Agent models and maps them to
the right providers. Apps can now use `openai/gpt-5.4-mini`,
`openai/gpt-5.5`, and `anthropic/claude-haiku-4-5` via the public API.
- **New Features**
- Registered the models in `AVAILABLE_CUA_MODELS` and mapped providers:
`gpt-5.4-mini` → `openai`, `gpt-5.5` → `openai`, `claude-haiku-4-5` →
`anthropic`.
- Updated public API tests to include the new models.
- Added a changeset to release these additions as a patch in
`@browserbasehq/stagehand`.
<sup>Written for commit ec5b691fdf9e469e11b4d025a11a7ac32dbba0c5.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/2065?utm_source=github">Review
in cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
---------
Co-authored-by: Chromie <miguel@browserbase.com>
Co-authored-by: Claude <noreply@anthropic.com>