109 Commits

Author SHA1 Message Date
Nick Sweeting
424efa1800
Real FlowLogger PR (several big commits were missing from #1824) (#1836)
# why

This is the real, recovered FlowLogger PR + improvements based on
yesterday's review session.

I accidentally force-pushed an earlier, incomplete branch state onto the
original flow logger PR and dropped several days of work. This PR
replaces that broken state with the actual recovered branch and the
follow-up fixes needed to make the design correct again.

<img width="1569" height="1015" alt="image"
src="https://github.com/user-attachments/assets/f99171fe-27df-488c-a87a-75605c2a3c1b"
/>


## what this PR does

This restores the intended v3 FlowLogger/EventStore design:

- every `V3` instance owns exactly one FlowLogger session
- every `V3` instance owns exactly one `EventStore`
- FlowLogger produces structured `FlowEvent`s on the instance bus
- EventStore consumes those events and fans them out to sinks
- the default query history is shallow and bounded so logging does not
grow memory without limit

The goal is a stable, queryable execution tree for
Stagehand/agent/CDP/LLM work that preserves parent/child relationships,
survives ALS re-entry, and does not let sink failures break main
execution.

## architecture

### FlowLogger

`packages/core/lib/v3/flowlogger/FlowLogger.ts`

FlowLogger owns:

- the `FlowEvent` model
- the `FlowLoggerContext` stored in AsyncLocalStorage
- method/closure wrappers: `wrapWithLogging(...)`, `runWithLogging(...)`
- re-entry helpers: `withContext(...)`, `resolveContext(...)`
- CDP event helpers
- LLM request/response helpers and middleware

FlowLogger is the event producer. It is responsible for building the
execution tree and maintaining the active parent stack. It is not
responsible for persistence or output destinations.

### EventStore

`packages/core/lib/v3/flowlogger/EventStore.ts`

EventStore owns:

- per-session sink registration
- query routing
- file-backed session directory setup
- bounded in-memory ancestry retention
- store lifecycle and teardown

Sink implementations live in
`packages/core/lib/v3/flowlogger/EventSink.ts`, and
prettifying/sanitization lives in
`packages/core/lib/v3/flowlogger/prettify.ts`.

EventStore is the consumer/router layer. It is intentionally
per-instance and single-session.

### V3 wiring

`packages/core/lib/v3/v3.ts`

In the `V3` constructor:

- `this.eventStore = new EventStore(this.sessionId, opts)`
- `this.flowLoggerContext = FlowLogger.init(this.sessionId, this.bus)`
- `this.bus.on("*", this.eventStore.emit)`

So FlowLogger always has a per-instance context, and the bus is the
handoff point between FlowLogger and EventStore.

## lifecycle

### FlowLogger lifecycle

1. `V3` creates the per-instance `FlowLoggerContext`
2. instrumented methods enter through `wrapWithLogging(...)` or
`runWithLogging(...)`
3. a started event is emitted
4. that event is pushed onto `parentEvents`
5. nested work inherits that parent stack through ALS
6. completed/error events are emitted on unwind
7. `V3.close()` clears the in-memory flow context

### EventStore lifecycle

1. `V3` creates one `EventStore(sessionId, opts)`
2. the default shallow in-memory query sink is attached immediately
3. optional sinks are attached based on runtime config
4. `V3` forwards wildcard bus events into the store
5. `destroy()` tears down the configured sinks

## config / runtime options

### always-on pieces

Every `V3` instance always gets:

- FlowLogger context
- shared event bus
- one EventStore
- one default shallow in-memory query sink

So instrumentation exists even if no human-visible sink is active.

### stderr sink

Enabled automatically when:

- `verbose: 2`
- `BROWSERBASE_FLOW_LOGS=1`

Behavior:

- writes prettified logs to stderr
- suppresses CDP events to keep interactive output high-signal
- does not retain history
- is best-effort only

### file sinks

Enabled automatically when:

- `BROWSERBASE_CONFIG_DIR` is set

Behavior:

- `JsonlFileEventSink` writes full events to `session_events.jsonl`
- `PrettyLogFileEventSink` writes prettified lines to
`session_events.log`
- `session.json` is written with sanitized options
- `sessions/latest` is maintained best-effort
- file sinks are write-only; `query()` returns `[]`

### queryable in-memory history

Default behavior:

- EventStore attaches a `ShallowInMemoryEventSink`
- retention limit is `500` events per session
- retained events keep ancestry metadata only
- retained `data` is intentionally stripped in the default sink

That shallow history is what pretty formatting uses to recover recent
ancestry without retaining screenshots/base64 payloads forever.

## memory impact

The main memory constraint is the default query sink.

What it retains:

- up to 500 recent shallow events per `V3` instance
- ids, timestamps, event types, parent ids, session id

What it does not retain by default:

- full screenshots
- large base64 blobs
- full historical payloads

Implications:

- memory scales roughly with `number_of_sessions * 500 shallow events`
- stderr sink does not retain history
- file sinks do not materially retain history in process memory beyond
normal stream buffering
- attaching `InMemoryEventSink` explicitly is an opt-in to full-payload
retention

## usage patterns

### use `wrapWithLogging(...)`

For class methods that should emit their own started/completed/error
envelope:

- `V3.act`
- `V3.extract`
- `V3.observe`
- page/understudy methods
- agent execute entrypoints

### use `runWithLogging(...)`

For closures or non-decorator code paths that still need a lifecycle
envelope.

### use `withContext(...)` / `resolveContext(...)`

For callback-driven or later async work:

- websocket callbacks
- CDP response/message dispatch
- detached async continuations

`currentContext` is strict and may throw when ALS is missing.
`resolveContext(...)` is the non-throwing lookup for “active ALS if
present, otherwise instance-owned fallback”.

### LLM logging is best-effort

If no flow context is active, LLM logging should no-op. Missing context
must not break model execution.

## reviewer expectations / edge cases

Please review this PR with these expectations in mind:

- ALS can legitimately be missing in later callbacks or detached async
work; that should not break execution unless a caller explicitly uses
strict `currentContext`
- there should be no global or multi-session fallback store; logging
belongs to a real `V3` instance/session
- parent-chain correctness matters more than sink behavior; events must
land under the right parent chain
- sink failures should not break the main library; sinks are
intentionally best-effort
- completion/error pretty lines should resolve back to the original
started-event ancestry, not synthetic completion ids
- `sessionId` often matches `browserbaseSessionId` today, but code
should not rely on that as a permanent invariant
- this PR is about FlowLogger/EventStore architecture and correctness,
not a model-default migration

---------

Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-03-17 15:02:12 -07:00
Sean McGuire
ac4539ab6b
[chore]: refactor & fix lint for browse CLI (#1821) 2026-03-12 17:44:59 -07:00
Nick Sweeting
ab2a05b041
Fix GITHUB_TOKEN usage in external PR approval flow (#1812)
# why

bug in previous approach

# what changed

# test plan

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes the external PR approval flow by switching to the correct
`GITHUB_TOKEN`, stabilizing the mirror/refresh behavior, and ignoring
third‑party bot comments when parsing claim markers. Also improves the
`claude` workflow to build the repo before edits and allow rerunning
failed jobs.

- **Bug Fixes**
- Use `GITHUB_TOKEN` for branch pushes and API calls; remove the GitHub
App token path.
  - Enable `persist-credentials: true` during checkout to allow pushes.
- Keep the mirrored PR open and mark it stale when new commits land on
the external PR; relabel both PRs consistently.
- Auto-handle reopen/close transitions across external and mirrored PRs.
- Ignore comments from non-managed bots (e.g., Greptile, Cubic); only
parse claim markers from `github-actions[bot]` to avoid false triggers.

- **Refactors**
- Inline a small JS lib (`ECPR_LIB`) to manage labels, comments,
lifecycle, and claims; jobs run in clear phases (external lifecycle →
claim prep → branch refresh → claim finalize).
- Refresh internal branches by rebasing onto the approved external ref;
report conflicts cleanly for manual follow-up.
- Improve `claude.yml`: upgrade to `actions/checkout@v6`, set `actions:
write`, run `pnpm`/`turbo` build via `setup-node-pnpm-turbo`, enable
`track_progress`, and use an explicit tool allowlist for
`anthropics/claude-code-action@v1`.

<sup>Written for commit a46b159b8337097c3f744351f888c4a59d51029d.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1812">Review in
cubic</a></sup>

<!-- End of auto-generated description by cubic. -->
2026-03-11 16:59:14 -07:00
Nick Sweeting
4b71374827
[STG-1518] Add Github Action that allows claiming external contributor PRs to run CI with secrets (#1794)
# why

- External contributor PRs currently fail CI because they cant run with
secrets
- We dont want to allow them to run with secrets until a team member
"claims" them and reviews for any secrets exfiltration / sketchy code
- Once claimed, we want to run the full CI suite with secrets

# what changed

# test plan

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Adds two GitHub Actions that let maintainers claim external contributor
PRs by mirroring the approved head SHA to a maintainer-owned branch so
full CI can run with secrets. Claims come from an approving review by a
team member with write access on the latest commit and are
auto-invalidated on new commits (Linear STG-1518).

- **New Features**
- Detects forked PRs and posts claim instructions; manages labels:
`external-contributor`, `external-contributor:awaiting-approval`,
`external-contributor:mirrored`, `external-contributor:stale`,
`external-contributor:completed`.
- On approving review of the latest commit, verifies reviewer
permission, mirrors that exact SHA to
`external-contributor-pr-<PR#>-<12sha>`, and creates/reopens a “[Claimed
#X]” PR assigned to the approver.
- Closes and links the original PR with marker comments; keeps
labels/status in sync on both PRs.
- Auto-closes the mirror when new commits land on the external PR and
comments with next steps; if the mirror closes without merge, reopens
and relabels the original PR; if the external PR is reopened with the
same approved SHA while the mirror is open, it is closed again to keep
discussion on the mirror.
- Implemented via `external-contributor-pr-approval-handoff.yml`
(captures approved reviews, uploads artifact) and
`external-contributor-pr.yml` (consumes artifact, performs mirroring);
uses `actions/github-script@v7`, `actions/create-github-app-token@v1`,
`actions/checkout@v4`, `actions/download-artifact@v4`,
`actions/upload-artifact@v4`; concurrency scoped per PR/workflow run.

- **Migration**
- Create a GitHub App with `contents:write`, `pull_requests:write`, and
`issues:write`; add `EXTERNAL_CONTRIBUTOR_PR_APP_ID` and
`EXTERNAL_CONTRIBUTOR_PR_APP_PRIVATE_KEY` secrets.
- To claim: submit an approving review on the latest commit of a forked
PR. If new commits are pushed, approve again to re-claim and rerun CI.

<sup>Written for commit 4875e99932a2f114cfeac184b0612ae2d683f9e1.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1794">Review in
cubic</a></sup>

<!-- End of auto-generated description by cubic. -->
2026-03-11 14:59:39 -07:00
Shrey Pandya
e38c13b752
[feat]: add browse CLI package (#1793)
## Summary
- Adds the `@browserbasehq/browse-cli` package (`packages/cli`) to the
stagehand monorepo, open-sourcing browser automation for AI agents
- CLI provides stateful browser control via a daemon architecture —
navigation, clicking, typing, screenshots, accessibility snapshots,
multi-tab, network capture, and env switching (local/remote)
- Uses `@browserbasehq/stagehand` as a workspace dependency (bundled
into the CLI binary via tsup)
- Includes full test suite and documentation

## Changes
- `packages/cli/` — all CLI source code, config, tests, and docs
- `pnpm-workspace.yaml` — added `packages/cli` to workspace
- `.github/workflows/ci.yml` — added CLI path filters and build artifact
uploads
- `.changeset/open-source-browse-cli.md` — changeset for initial release
- `pnpm-lock.yaml` — updated lockfile

## Test plan
- [x] CLI builds successfully (`pnpm --filter @browserbasehq/browse-cli
run build`)
- [x] Full monorepo build passes (`turbo run build` — 9/9 tasks)
- [x] `browse --help` and `browse --version` output correctly
- [x] `browse status` returns valid JSON
- [x] Lint passes clean (`pnpm --filter @browserbasehq/browse-cli run
lint`)
- [x] Source verified identical to stagent-cli (only import path
changed)
- [x] Empirically tested Browserbase credential requirements match core
- [ ] Run `pnpm --filter @browserbasehq/browse-cli run test` (requires
Chrome/browser environment)

## Known issues (pre-existing from stagent-cli, not introduced by this
PR)
- Network capture `response.json` always writes `status: 0` — response
metadata from `responseReceived` CDP event is not persisted to
`loadingFinished` handler
- Ref-based `click` command silently ignores
`--button`/`--count`/`--force` flags (coordinate-based `click_xy`
handles them correctly)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-09 17:56:45 -07:00
Nick Sweeting
e36ddd6f29
[STG-1536] Split packages/stagehand/server into separate server-v3 and server-v4 dirs (#1796)
# Follow-up Tasks

- [ ] Update stainless SDK custom code for all languages to pull new
`stagehand-server-v3-darwin-x64` binary names (`-v3-` added)

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Split the Stagehand API into `packages/server-v3` and
`packages/server-v4`, each with its own builds, tests, SEA binaries, and
release workflows. Delivers STG-1536 and lets us keep v3 stable while
iterating on v4; CI/test discovery and OpenAPI artifacts are versioned.

- **Refactors**
- Renamed the original server to `packages/server-v3`
(`@browserbasehq/stagehand-server-v3`); updated docs and runtime path
helpers (now synced across core/docs/evals and both servers), ESLint
globs/ignores, scripts/Turbo filters, tests, and Stainless to read
`packages/server-v3/openapi.v3.yaml`; v3 SEA binaries use
`stagehand-server-v3-*`.
- Added `packages/server-v4` (`@browserbasehq/stagehand-server-v4`) with
`/v4/**` routes, SSE streaming via `x-stream-response`, LRU/TTL
in-memory session store, health/readiness, logging/metrics,
`openapi.v4.yaml` + generator, SEA tooling, and v4 integration tests.
- CI: path filters, test discovery, and artifacts cover both versions;
added `stagehand-server-v4-release.yml` and
`stagehand-server-v4-sea-build.yml`; renamed v3 workflows; artifacts
include `packages/server-v3/**` and `packages/server-v4/**` dists and
OAS.

- **Migration**
- Replace `packages/server/**` refs with `packages/server-v3/**` or
`packages/server-v4/**`.
- Use new package filters and binary names:
`@browserbasehq/stagehand-server-v3` /
`@browserbasehq/stagehand-server-v4`; `stagehand-server-v3-*` /
`stagehand-server-v4-*`.
- Update OpenAPI consumers to `packages/server-v3/openapi.v3.yaml` or
`packages/server-v4/openapi.v4.yaml`.

<sup>Written for commit 2b9114ca8097d28ac1567fc67062b24f789a5c2e.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1796">Review in
cubic</a></sup>

<!-- End of auto-generated description by cubic. -->
2026-03-09 17:32:57 -07:00
Nick Sweeting
9df8fe66ad
Revert "[fix]: skip CI jobs requiring secrets on fork PRs" (#1787)
Reverts browserbase/stagehand#1780

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Reverts the change that skipped CI on forked PRs. Integration tests,
evals, and the Stainless preview now run for all PRs by removing the
head-repo equality checks in ci.yml and stainless.yml.

<sup>Written for commit 18480e852cf5c3937f22994450c88b169a4fa6c8.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1787">Review in
cubic</a></sup>

<!-- End of auto-generated description by cubic. -->
2026-03-04 17:25:46 -08:00
Nick Sweeting
748a673d17
Revert "Allow outside contributors to run CI with secrets when approved" (#1786)
Reverts browserbase/stagehand#1782

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Reverts the approval-based CI for external contributors. CI now runs on
pull_request and blocks secrets for forked PRs by skipping integration,
E2E, and eval jobs.

- **Refactors**
  - Removed the “Ensure Contributor Is Trusted to Run CI” workflow.
  - Switched CI trigger to pull_request; removed workflow_run logic.
  - Read labels from github.event.pull_request; removed API calls.
  - Simplified checkouts; dropped explicit head_sha refs.
  - Updated concurrency group to use github.ref.
  - Ignored docs-only changes in CI.

<sup>Written for commit d6ace82b40a20d8a821f210cea018adc09666f37.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1786">Review in
cubic</a></sup>

<!-- End of auto-generated description by cubic. -->
2026-03-04 17:20:32 -08:00
Nick Sweeting
02c31ef87e
Allow outside contributors to run CI with secrets when approved (#1782)
PSA potential hackers: dont get excited, we don't have any real secrets
in CI worth stealing, and our CI does not autodeploy anything to prod.
All important secrets and CD processes are kept in our closed-source
repos.

# why

# what changed

# test plan


<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Add a gating workflow that blocks CI until a maintainer approves running
secrets on forked PRs. CI now triggers from that gate, resolves labels
and path filters under workflow_run, removes same-repo guards so
integration/e2e/evals run on approved forks, and checks out the PR
commit consistently across jobs.

<sup>Written for commit c68284700554566b199875de2b2c901caedf8d6a.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1782">Review in
cubic</a></sup>

<!-- End of auto-generated description by cubic. -->
2026-03-04 16:55:51 -08:00
Shrey Pandya
37aeb68738
[fix]: skip CI jobs requiring secrets on fork PRs (#1780)
## Summary
- Server integration tests, evals, and Stainless preview builds require repo secrets that GitHub doesn't expose to fork PRs
- These jobs were running and failing with missing env var errors on every fork PR
- Add the same fork guard (`head.repo.full_name == github.repository`) that e2e tests already use

### Jobs fixed:
- `server-integration-tests` in `ci.yml`
- `run-evals` in `ci.yml`
- `preview` in `stainless.yml`

## Test plan
- [ ] Verify existing (non-fork) PRs still run all CI jobs
- [ ] Verify fork PRs skip the guarded jobs gracefully

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Skip CI jobs that require repo secrets on fork PRs to prevent missing env errors. These jobs now run only when the PR comes from this repo.

- **Bug Fixes**
  - Guarded server integration tests in ci.yml.
  - Guarded eval runs in ci.yml.
  - Guarded Stainless preview builds in stainless.yml.

<sup>Written for commit f71de8d1f56fd156ac28704d07803c81ccf6b944. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1780">Review in cubic</a></sup>

<!-- End of auto-generated description by cubic. -->
2026-03-04 10:42:44 -08:00
Shrey Pandya
a110e39265
chore: remove retired Claude 3.5 and 3.7 Sonnet models (#1775)
Fixes CE-731

## Summary
- Remove Claude 3.5 Sonnet (`claude-3-5-sonnet-latest`, `-20241022`, `-20240620`) and Claude 3.7 Sonnet (`claude-3-7-sonnet-latest`, `-20250219`) from all supported model lists
- These models are **retired** by Anthropic — API calls to them will fail
- Replace with `claude-sonnet-4-20250514` across evals, CI, docs, and examples

## What changed (27 files)
- **Core types**: Removed from `model.ts` type union, `agent.ts` CUA models list
- **Provider mappings**: Removed from `LLMProvider.ts`, `AgentProvider.ts`, server `utils.ts`, server `model.ts`
- **Evals/CI**: Updated `taskConfig.ts`, `initV3.ts`, `ci.yml`, `.env.example` to use `claude-sonnet-4-20250514`
- **Tests**: Updated `model-deprecation.test.ts` and `llm-and-agents.test.ts` (513/513 pass)
- **Docs**: Updated all v2 and v3 documentation references (11 `.mdx` files)
- **Other**: Issue template, MCP example

## Context
Per [Anthropic's model deprecations page](https://docs.anthropic.com/en/docs/resources/model-deprecations):
| Model | Retired |
|-------|---------|
| `claude-3-5-sonnet-20240620` | Oct 28, 2025 |
| `claude-3-5-sonnet-20241022` | Oct 28, 2025 |
| `claude-3-7-sonnet-20250219` | Feb 19, 2026 |

## Test plan
- [x] All 513 unit tests pass (`pnpm exec turbo run test:core`)
- [x] `grep` confirms zero remaining references outside CHANGELOG.md (historical)
- [ ] Verify CI passes

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-03-03 10:29:50 -08:00
Nick Sweeting
73905088c5
[STG-1450] Fix Session init failing when no pages are open, falls back to opening about:blank (#1749)
# why

Stagehand API failures were caused on all routes `/act`, `/observe`,
etc. when session init failed because a page was closed. This adds a
fallback to open about:blank so at least the session remains usable
instead of just crashing on init.

# what changed

# test plan

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes session init when connecting to a browser with zero pages by
auto-creating an about:blank tab. Keeps Stagehand routes working after
the last page closes and surfaces lint failures via a dedicated
cancellation job. Addresses STG-1450.

- **Bug Fixes**
- Replace waitForFirstTopLevelPage with ensureFirstTopLevelPage: check
for an existing page, wait with timeout, then create about:blank if
none.
- Simplify errors to a TimeoutError with clearer context; improve
logging and add an integration test for zero-page init.

- **CI**
- Run cancellation in a separate job only when lint fails, so failures
are visible instead of marked skipped.

<sup>Written for commit 45ed6dc059d44f07c338dc8f36e72eb0702a2cfd.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1749">Review in
cubic</a></sup>

<!-- End of auto-generated description by cubic. -->

<!-- mintlify-editor-comments:start -->
Mintlify
---
0 threads from 0 users in Mintlify

- No unresolved comments
<!-- mintlify-editor-comments:end -->

<!-- mintlify-comment-->

<a
href="https://dashboard.mintlify.com/browserbase/stagehand/editor/session-init-fixes?source=pr_comment"
target="_blank" rel="noopener noreferrer"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://d3gk2c5xim1je2.cloudfront.net/assets/open-mintlify-editor-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://d3gk2c5xim1je2.cloudfront.net/assets/open-mintlify-editor-light.svg"><img
src="https://d3gk2c5xim1je2.cloudfront.net/assets/open-mintlify-editor-light.svg"
alt="Open in Mintlify Editor"></picture></a>

<!-- /mintlify-comment -->
2026-02-24 17:42:42 -08:00
Nick Sweeting
4f8d2d5d85
Fix Windows SEA build (#1734)
# why

# what changed

# test plan

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes the Windows SEA build and cross‑platform path resolution
(including when run via tsx). CI builds, launches, and smoke‑tests
Linux, macOS, and Windows binaries (x64/arm64); CJS/ESM outputs are
stable.

- Bug Fixes
- Reusable SEA build workflow with a full OS/arch matrix (incl. Ubuntu
24.04 ARM and Windows 11 ARM), same‑arch launch check against /healthz,
selective artifact upload, and an include_sourcemaps toggle (dev/test
only; releases exclude sourcemaps).
- Centralized runtimePaths helpers across core/server/evals/docs to
replace import.meta/fileURLToPath, fix tsx path resolution, normalize
absolute paths, and correct supervisorClient CLI resolution and Windows
spawning.
- Tooling: split DOM script builds into four Turbo tasks for Windows;
switch to in‑process esbuild and Node‑invoked turbo/postject; stabilize
CJS/ESM via tsc emits plus small CJS wrappers; refine tsconfig includes;
add default NODE_ENV/BB_ENV in server env.

<sup>Written for commit b0b191f7b0411b6590a79ce3f10b3fba9435b4d7.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1734">Review in
cubic</a></sup>

<!-- End of auto-generated description by cubic. -->

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Touches build/CI and packaging code (SEA generation, path resolution)
across multiple packages, so failures could break releases or CI on
specific platforms despite being largely tooling-focused.
> 
> **Overview**
> Fixes cross-platform SEA building by moving `build-server-sea` to a
reusable workflow with a multi-OS/arch matrix (including ARM runners),
optional artifact/caching inputs, selective artifact upload, and a
same-arch smoke test that boots each binary and hits `/healthz`.
> 
> Standardizes path/root resolution via new `runtimePaths` helpers
(core/server/evals/docs) and replaces ad-hoc
`fileURLToPath(import.meta.url)`/`__dirname` usage across scripts,
tests, and `supervisorClient` (also tightening CLI resolution and SEA
detection).
> 
> Simplifies build/tooling: core CJS/ESM builds now rely on `tsc` emits
plus small CJS wrapper entrypoints (dropping esbuild bundling of
core/tests), DOM script generation is split into separate Turbo tasks,
server SEA build switches to in-process `esbuild` and Node-invoked
`turbo`/`postject`, and server env config now provides default
`NODE_ENV`/`BB_ENV` values.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
2e4a07a6d0b54eddb7fea10d079c8062ecba1d6b. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-02-24 10:39:45 -08:00
Nick Sweeting
6329982da2
Fix typechecking for core/tests, fix SEA build for windows, speed up CI (#1721)
# why

# what changed

# test plan

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes TypeScript across core tests/scripts, finalizes the default
export/CLI, and makes the SEA build reliable on Windows. CI splits build
and lint, OpenAPI generation no longer runs Prettier, and builds run in
parallel across packages for faster pipelines.

- **Bug Fixes**
- Stricter TS across tests/scripts (tsconfig includes tests + CLI, added
@types/jsdom, switched to playwright-core); tightened types (child-frame
probes, DocumentReadyState, model utils, source-map Raw types); fixed
OpenAPI typing; removed cross-package imports in server tests; added
default-export parity test.
- Default export and CLI fixed: introduced StagehandDefault mirroring
named exports; ESM/CJS indices now export default; corrected
package.json CLI paths/types.
- SEA build works on Windows: build scripts use pnpm.cmd with better
spawn error handling.
- List parsing now works from raw source across core/server/evals and
maps cleanly to dist outputs.

- **Refactors**
- CI split into separate Build and Lint jobs; lint cancels the workflow
on failure; optional Turbo cache restore; discovery/runner steps no
longer depend on the build job.
- Parallelized builds: DOM script generators run concurrently; ESM/CJS
builds and server/evals build tasks run in parallel; tuned Turbo inputs
to reduce noisy rebuilds and improve caching.

<sup>Written for commit a25e17c2411b374e8d35c39f6120d242102ff8dc.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1721">Review in
cubic</a></sup>

<!-- End of auto-generated description by cubic. -->

---------

Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-20 17:09:39 -08:00
Nick Sweeting
50a6f574a3
Speed up turbo, package.json scripts, and CI configuration (#1712)
# why

# what changed

# test plan

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Speed up builds and tests by upgrading Turborepo, tightening cache keys,
and refactoring scripts to run from compiled dist (ESM/CJS). CI is
faster and more reliable, with ESM-first SEA builds for the server.

- **Refactors**
- Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit
inputs/outputs for better caching.
- Stronger Turbo cache keys in CI (lockfiles + workspace + package.json
+ turbo.json + commit), workflow-scoped concurrency, and a 60s session
create timeout.
- Standardized CI via a composite setup action; build now runs before
lint.
- Build/test scripts resolve the repo root via fileURLToPath; tests run
from compiled dist; add Vitest config for CJS; CJS builds include
sourcemaps.
- Server: ESM-first SEA build and split server build into dist vs SEA
steps with a new build:server:dist script.
- Unified test scripts across packages; top-level test runs all suites,
and evals tests support source or dist CLI via --cli.

- **Migration**
  - Use Node 20.19+ or 22.12+ (enforced in core).
  - Run pnpm install to pick up the Turbo upgrade.
- Build before tests: pnpm -w build. For SEA tests, also run pnpm -w
--filter=@browserbasehq/stagehand-server run build:sea:esm.

<sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in
cubic</a></sup>

<!-- End of auto-generated description by cubic. -->

---------

Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
Sean McGuire
a506b1634b
[chore]: rm duplicate test runs (#1704)
# why
- for now, we only really need to run tests on pull requests. rerunning
on merge to main is redundant
# what changed
- reverted the CI trigger to only run on pull requests

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Run tests only on pull requests to prevent duplicate CI runs on merges
to main. This reduces CI noise and usage.

Updated the GitHub Actions workflow to remove the main push trigger,
drop main-specific test logic, simplify doc-only skip rules, and run e2e
jobs only for PRs from this repo.

<sup>Written for commit 592ad4441c3d2f5d9ccda9f53c4b678fca27ab90.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1704">Review in
cubic</a></sup>

<!-- End of auto-generated description by cubic. -->
2026-02-18 14:37:53 -08:00
Sean McGuire
2d6599c5fe
[chore]: re add registry-url to release.yml (#1703)
# why
- to fix alpha releases which were failing 
# what changed
- this PR adds back the the registry-url config to the `setup-node`
step, which is required for trusted publishing


<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Restores npm registry config and updates the Node setup step to v6 for
Trusted Publishing, fixing failing alpha releases.

- **Bug Fixes**
- Configure Trusted Publishing: set registry-url via setup-node v6 with
Node 20.x and upgrade npm to latest.
  - Set checkout fetch-depth to 0 to include full git history.

<sup>Written for commit 90123dac483dde9c890c8d754179945ae2bb16a6.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1703">Review in
cubic</a></sup>

<!-- End of auto-generated description by cubic. -->
2026-02-18 14:32:29 -08:00
Nick Sweeting
afbd08bb63
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632)
# why

- unit tests, e2e integration tests, and evals were taking 10~15 minutes
per pr
- finding the specific failing eval or test was tedious and required
reading tons of log output
- we weren't leveraging pnpm or turbo caches or github action's native
chromium install
- tests were flaky because CDP would randomly fail to connect when
github actions runner was overloaded
- e2e bb tests were not actually running against bb, so it was
previously just double-running e2e:local and reporting it as bb
- tsup has been deprecated for a while and should be replaced by esbuild

# what changed

- parallelized all CI tests and broke out evals and tests into
individual matrix jobs
- created unified node+pnpm+turbo setup action that all other actions
call
- verified vanilla chromium is able to launch inside all the e2e local
jobs before running any of our own code to catch resource contention
flakyness
- add proper esm and csj builds with sourcemaps and types for aligned
coverage
- remove tsup dependency
- switch to new evals cli
- add coverage reporting
- fix some flaky tests needed to get everything passing reliably

<img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM"
src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7"
/>



<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Speeds up CI and makes tests stable by running Vitest/Playwright/Evals
on the built ESM dist with cached artifacts, verified runner Chromium,
weighted Browserbase routing, and merged coverage. Publishes a unified
ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes
Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test
reporting, addressing Linear STG‑1232.

- **New Features**
- CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium
launch check, region selection, CTRF/V8 uploads); reuse artifacts; use
Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports;
V8 coverage normalization/merge with cancellation safety; push triggers
enabled.
- Tests: run from dist/esm; env reporter; true Browserbase runs with
weighted/randomized regions; hard timeouts for LLM and session
create/CDP connect; faster multi‑click dispatch; higher timeouts and
targeted skips; cleanup helpers; fixes for init‑script,
wait‑for‑selector, and screenshot reliability.
- Build/SDK/SEA: export map (ESM import + CJS require) with
sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain
logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and
exported; getAISDKLanguageModel added; replay metrics now include
method/parameters/result/timestamps/cost.

- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV
(replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS,
BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS;
optionally set BROWSERBASE_REGION_DISTRIBUTION.
- Remove dotenv; set env vars directly and import
@browserbasehq/stagehand via the export map (avoid dist paths).

<sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>

<!-- End of auto-generated description by cubic. -->







<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types.
CI runs Vitest/Playwright and the new Evals CLI against the built ESM
dist using the runner’s Chromium (launch‑verified), with region‑weighted
Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and
more stable tests (timeouts, Browserbase skips, understudy click/API
fixes).

Speed up CI by running Vitest and Playwright tests in parallel matrices,
add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard
timeouts and better cleanup. Server integration tests now use the
runner’s Chromium for reliability and are folded into the main CI.

- **Refactors**
- Parallelized core unit tests via discover-core-tests/core-unit-tests
(Vitest matrix, fail-fast).
- Parallelized E2E via discover-e2e-tests (Playwright local +
Browserbase) gated on core changes; weighted Browserbase region
selection; verify Chromium/CDP before tests; local uses runner Chromium
via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when
CTRF_JUNIT_PATH is set.
- Unified Lint & Build with Turbo caching and shared artifacts;
tests/evals download and reuse; prepare script skips build in CI; SEA
builds support sourcemaps.
- Enforced configurable timeouts and parallelism via env (LLM_MAX_MS,
BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS,
LOCAL_SESSION_LIMIT_PER_E2E_TEST,
BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and
Browserbase hard timeouts; improved click event dispatch to reduce
remote flakiness.
- Explicitly close Browserbase sessions in tests and evals to free
capacity.
- Added composite actions: setup-node-pnpm-turbo,
verify-chromium-launch, select-browserbase-region, publish-ctrf-report
(JUnit→CTRF and evals), upload-v8-coverage.
- Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter
for CI logs; removed the standalone stagehand-server tests workflow and
ran server tests in the unified CI with Node’s test runner and GitHub
Actions Chromium.


- **New Features**
- Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map;
types from ESM; SEA binaries built from dist with sourcemaps; server SEA
job targets CJS; new build/test/coverage scripts; Node loader maps
package imports to dist/esm; ESM‑safe pathing (fileURLToPath);
ESLint/Prettier tweaks; dotenv removed.
- CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch,
select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses
runner Chromium via CHROME_PATH with --no-sandbox and launch checks;
weighted Browserbase region selection; unique CTRF artifacts; V8
coverage normalized/merged with debug logs and graceful cancellation;
Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled.
- Tests & API: Vitest/Playwright run on dist/esm; new env reporter and
closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and
targeted skips (e.g., console on Browserbase) reduce flakiness; local
launches honor CHROME_PATH; replay metrics now include
parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI
clients moved to lib/v3/external_clients and exported; env‑driven hard
timeouts for LLM ops and Browserbase session create/CDP connect;
understudy API usage fixes.

- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune
LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and
BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set
BROWSERBASE_REGION_DISTRIBUTION.
  - dotenv support is removed; set env vars explicitly in dev and CI.
- Import @browserbasehq/stagehand via the export map; do not reference
dist paths or prior tsup outputs.

<sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>

<!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
Miguel
653268b1b3
fix(ci): handle new tags in server release workflow (#1660) (#1661)
# why
The git fetch command was failing with exit code 128 when trying to
fetch a tag that doesn't exist on the remote yet (which is the case for
new releases). This caused the entire release workflow to fail.

# what changed
Added `|| true` to allow the fetch to fail gracefully for new tags. The
subsequent git rev-parse correctly handles both cases:
- Existing tag: script exits with "Tag already exists" message
- New tag: script proceeds to create and push the new tag

# test plan

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Prevented the server release workflow from failing on new releases by
letting git fetch ignore missing remote tags. Existing tags exit with a
clear message; new tags proceed to be created and pushed.

<sup>Written for commit d71339d227d18e96d9fdca926ebc966808e00464.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1661">Review in
cubic</a></sup>

<!-- End of auto-generated description by cubic. -->

Co-authored-by: Chromie <miguel@browserbase.com>
Co-authored-by: Chromie Bot <chromie@browserbase.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 13:17:43 -08:00
Miguel
e5009fce98
Fix SEA release workflow for Windows (#1603) 2026-01-23 16:41:54 -08:00
Nick Sweeting
922f7ef42f
add v3 integration tests matching cloud exactly (#1583) 2026-01-20 19:03:12 -08:00
Sam F
a5074bdda0
End endpoint cleanup (#1578)
# why
`stagehand-api` was updated so that no request body should be send when
calling the `/end` endpoint.

# what changed
Thus, we can simplify our `stagehand-server` code so that no request
body should be sent as well, cleaning up our stainless sdks.

Also made it so that when we want to release a new github release of
`stagehand-server`, we add in a minor or major changeset - no longer
having to bump the `package.json` version manually.
# test plan
Added a minor changeset for `stagehand-server` and will confirm new
github release.

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Removed the empty JSON body requirement for POST /v1/sessions/:id/end.
The endpoint now accepts no request body, and the release workflow uses
Changesets instead of manual version bumps.

- **Refactors**
- Removed the SessionEnd request body from server route and OpenAPI; SDK
types now allow no body.
- Validate that the request body is empty; allow 0-byte JSON and return
400 for non-empty or non-object bodies. Tests assert 200 with no body or
empty JSON.
- Updated the GitHub release workflow to detect server releases via
Changesets (no package.json bump needed).
  - Added port to browser launch options in the OpenAPI spec.

- **Migration**
- Call POST /v1/sessions/:id/end with no body; sending Content-Type:
application/json with an empty body is allowed.

<sup>Written for commit 975ccb6da02a782078590424a44640f1a5b01d1c.
Summary will update on new commits.</sup>

<!-- End of auto-generated description by cubic. -->

---------

Co-authored-by: ci-test <ci-test@example.com>
2026-01-20 15:44:07 -08:00
Sam F
723a7fda91
/end endpoint returns empty object (#1500)
# why
Right now our server `/end` endpoint doesn’t match the production
Stagehand API contract: prod expects a non-empty JSON body when
`Content-Type: application/json` is set, so a call to `/end` that sends
an empty body fails. This change makes our server + OpenAPI spec match
prod so downstream generated SDKs send an empty object.

# what changed
Defined a strict empty-object request schema for `/end`
(`SessionEndRequest`) and made it a required request body in the route +
emitted OpenAPI. Updated the /end handler response to match prod `({
"success": true })` and added a server test covering the body
requirement.

- Also added a `stainless.yaml`
# test plan
Regenerate the Python SDK and verify sessions.end() succeeds against
production
2026-01-05 20:45:33 -07:00
Sam F
81434a7bf3
server binary fixes (#1474)
# why
Getting CI pipeline working to build server binaries
# what changed
Removed colons from fastify routes for windows compat, and updated so
that we're no longer using deprecated macos version for runner.
# test plan
See checkmarks in CI

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes CI builds for server SEA binaries by making route file paths
Windows-compatible and updating macOS runner versions.

- **Bug Fixes**
- Renamed route directories from v1/sessions/:id/* to v1/sessions/_id/*
and updated imports in server.ts and gen-openapi.ts to avoid colon
issues on Windows.

- **Dependencies**
- GitHub Actions now uses macos-15 and macos-15-intel runners (replacing
macos-latest and macos-13), and the build job name reflects binary_name.

<sup>Written for commit 526f95b658f5f8170eed6a984891b64dd8ceccbb.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-12-24 11:13:15 -07:00
Sam F
87eac334f6
No longer using changesets for stagehand-server - manually bumping package.json version number (#1470)
# why
Changesets requires waiting for release PRs and we want to push new
stagehand-server versions more frequently
# what changed
No longer using changesets - manually bumping package.json version
number
# test plan




<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Switched stagehand-server releases from Changesets to a dedicated
workflow that publishes on manual version bumps. This speeds up releases
and automates tagging, binaries, and GitHub Releases.

- **Refactors**
- Added .github/workflows/stagehand-server-release.yml to detect version
bumps, tag stagehand-server/vX.Y.Z, and publish a GitHub Release.
- Builds SEA binaries for Linux, macOS, and Windows (x64/arm64), and
uploads the OpenAPI spec and SHA256 checksums.
- Removed server release logic from .github/workflows/release.yml;
canary publish remains.
  - Bumped @browserbasehq/stagehand-server to 3.1.1.

- **Migration**
- To release: bump packages/server/package.json and merge to main (or
trigger the workflow manually).

<sup>Written for commit 21f2cc37de55d82e6a4b8c27ea94f16076da0021.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-12-23 16:31:25 -07:00
Sam F
d634d45a0d
Gate release creation on package.json version changes on main, instead of Changesets publishedPackages. (#1469)
# why
Server binaries/releases were skipped because
`@browserbasehq/stagehand-server` is private (not npm-published), so the
workflow never detected a “server publish” event.
# what changed
Gate server tagging + binary build + GitHub Release creation on
packages/server/package.json version changes on main, instead of
Changesets publishedPackages.
# test plan
See if binaries are build on merge to main



<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Switch server release workflow to trigger on
packages/server/package.json version changes on main. This fixes skipped
server binaries/releases for the private @browserbasehq/stagehand-server
package.

- **Bug Fixes**
- Compare server package version between before/after commits and set a
server_release flag.
- Gate tag creation, binary builds, and GitHub Release on
server_release.
- Annotate tag to the current SHA and rename outputs from
server_published to server_release.
  - Add a changeset bump to test the pipeline.

<sup>Written for commit e224b9510b79e6b576d8d4e68d466bbc49c3e370.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-12-23 15:37:07 -07:00
Sam F
54e4dc5b49
Moved server binary build into release.yaml (#1467)
# why
Fix server binaries not publishing because tag pushes from CI didn’t
trigger the separate tag-based workflow.
# what changed
Move server binary build + GitHub Release publishing into release.yml,
gated on Changesets publishing @browserbasehq/stagehand-server.
# test plan
See if binaries are published in release (can't test with act)


<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Moved the stagehand/server binary build and GitHub Release into the main
release workflow. Fixes missed server binary releases by gating on
Changesets publishing of @browserbasehq/stagehand-server.

- **Bug Fixes**
- Server binaries now publish reliably, triggered by Changesets publish
instead of CI tag pushes.

- **Refactors**
- Integrated server build and release into .github/workflows/release.yml
with detection outputs (server_published, server_version, server_tag).
- Added a cross-platform build matrix and a release job that attaches
the OpenAPI spec and SHA256 checksums.
  - Removed .github/workflows/stagehand-server-release.yml.

<sup>Written for commit 180a1c8280a659556773ca4ea4e27cdc4e084635.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-12-23 12:32:39 -07:00
Sam F
abb3469f51
Add code to publish binaries on release (#1459)
# why
Stagehand-python needs a stable, versioned place to fetch the exact
stagehand/server SEA binaries for packaging into wheels. Today our
stagehand-server/v* GitHub Releases only include the OpenAPI spec, so
the binary handoff isn’t automatic.

# what changed
`stagehand-server/v*` releases now build a cross-platform SEA binary
matrix (linux/macOS/windows) and attach the resulting binaries (plus
checksums) to the same GitHub Release. The release still includes the
versioned OpenAPI spec asset as before.

# test plan
Binaries should build on merge to main



<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Publish cross-platform stagehand-server SEA binaries on each versioned
GitHub Release, with SHA-256 checksums. This makes it easy for
stagehand-python to fetch exact, versioned server binaries.

- **New Features**
- CI builds SEA binaries for linux-x64, linux-arm64, darwin-arm64,
darwin-x64, win32-x64, and win32-arm64.
- Binaries and checksums are attached to the stagehand/server v* GitHub
Release alongside the OpenAPI spec.
- Added build scripts (build-binary.sh, create-binary.sh) and
sea-config.json; local cross-compiling is not supported.
- Artifacts are produced per-OS via a matrix job and collected in the
release.

- **Dependencies**
- Added esbuild and postject to support bundling and SEA blob injection.

<sup>Written for commit acc3bf5433fde74d3a589e69079e949b6eafbbdb.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-12-22 18:10:38 -07:00
Sam F
5fc12817a6
Pushing stagehand-server to github releases with changesets, instead of using github artifacts (#1457)
# why
With my previous artifacts approach, our client sdk repos like
`stagehand-python` just pulled the most recent artifact, which could be
error prone - by having it pull a versioned release, we avoid a swath of
potential issues.
# what changed
Added logic for pushing releases. 
# test plan
We should see a release show up for stagehand-server when this merges,
since I added a minor version bump changeset.

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Publish stagehand-server via versioned GitHub Releases driven by
Changesets, replacing the previous GitHub Artifacts flow. SDKs can now
fetch stable, versioned assets (OpenAPI spec) from releases.

- **New Features**
- Auto-tag stagehand-server/v{version} when packages/server version
changes; skips if unchanged or tag exists.
- New release workflow verifies the tag matches package.json, packages
openapi.v3.yaml, and publishes a GitHub Release with notes.
- Added a minor Changeset for @browserbasehq/stagehand-server to trigger
the first release.

- **Migration**
- Update SDKs (e.g., stagehand-python) to download
openapi.v3.stagehand-server-{version}.yaml from the matching GitHub
Release tag.
  - Stop relying on the “latest” GitHub Artifacts.

<sup>Written for commit e47c56e3c1a61c186dc01f867b61f7feba28d76d.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-12-22 14:50:15 -07:00
Sam F
8ee996ece7
Add Stainless CI for SDK codegen (#1450)
# why
Add Stainless CI for SDK codegen Keep all client SDKs in sync with
packages/server/openapi.v3.yaml automatically, with safe previews before
merge.

# what changed
Adds the Stainless-recommended GitHub Actions workflow: run preview@v1
on PR updates and merge@v1 when the PR is merged, using OIDC via the
Stainless GitHub App.
# test plan




<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Sets up Stainless CI in GitHub Actions to auto-sync SDKs from
packages/server/openapi.v3.yaml. Runs preview builds on PRs and a merge
build when PRs are merged.

- **New Features**
- Added .github/workflows/stainless.yml using Stainless preview@v1 on PR
updates and merge@v1 on PR merge.
- Uses API key auth via secrets.STAINLESS_API_KEY with least-privilege
permissions and run cancelation.
- Reads STAINLESS_ORG, STAINLESS_PROJECT, and OAS_PATH from Actions
Variables/env.
- Triggers on pull_request events (opened, synchronize, reopened,
closed).

- **Migration**
  - Add Actions Variables: STAINLESS_ORG and STAINLESS_PROJECT.
  - Set secrets.STAINLESS_API_KEY.
  - Ensure the spec lives at packages/server/openapi.v3.yaml.

<sup>Written for commit 9e18a31c14be78e9c58493605c600532d3c58001.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-12-22 10:53:38 -07:00
Sam F
fdbb58ce7a
Add fastify server (#1384)
# why
Stand up a Fastify Stagehand server we can reuse for thin-client SDKs
across multiple languages.

 # what changed
created new fastify server
# test Plan
- Start the Fastify server (pnpm --filter server dev or your usual
command).
- Local browser smoke: MODEL_API_KEY=... ./scripts/test_local_browser.sh
- Browserbase smoke: MODEL_API_KEY=... BROWSERBASE_API_KEY=...
BROWSERBASE_PROJECT_ID=... ./scripts/test_remote_browser.sh.
















<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Adds a new Fastify-based Stagehand API server exposing V3 browser
automation over REST with streaming responses and session management.
Supports both local Chrome and Browserbase, includes health/readiness
endpoints, and ships an OpenAPI spec.

- **New Features**
- New packages/server with REST routes: start, navigate, observe, act,
extract, agentExecute, end (streaming logs/results)
- In-memory LRU session store with TTL, lazy V3 init, and cleanup on end
  - Local and Browserbase browsers; credentials passed via headers
- Health (/healthz) and readiness (/readyz), metrics, and structured
request logging
  - OpenAPI v3 spec and README
  - Removed v2 code and DB dependency; auth currently disabled

- **Migration**
  - Run: pnpm --filter @browserbasehq/stagehand-server dev
- Required header: x-model-api-key; for Browserbase also x-bb-api-key
and x-bb-project-id

<sup>Written for commit ed1089bae65c7ea337e2012eb64efc664b36141c.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-12-09 15:51:10 -08:00
Nick Sweeting
898e1f438a
Remove unecessary claude PR review bot (#1361)
Keeps `@\claude` support but drops the auto PR reviews by claude, we
already have plenty of auto-review feedback from greptile and cubic.

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Removed the Claude PR review GitHub Action to stop automatic reviews.
Keeps @claude support and reduces duplicate bot feedback, since greptile
and cubic already provide auto-review.

<sup>Written for commit 740d927c7b0e0066af7786fd8e80a7c9e6fb6249.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-12-04 13:42:28 -08:00
Nick Sweeting
1c5c233790
Add Claude Code GitHub Workflow (#1348)
## 🤖 Installing Claude Code GitHub App

This PR adds a GitHub Actions workflow that enables Claude Code
integration in our repository.

### What is Claude Code?

[Claude Code](https://claude.com/claude-code) is an AI coding agent that
can help with:
- Bug fixes and improvements  
- Documentation updates
- Implementing new features
- Code reviews and suggestions
- Writing tests
- And more!

### How it works

Once this PR is merged, we'll be able to interact with Claude by
mentioning @claude in a pull request or issue comment.
Once the workflow is triggered, Claude will analyze the comment and
surrounding context, and execute on the request in a GitHub action.

### Important Notes

- **This workflow won't take effect until this PR is merged**
- **@claude mentions won't work until after the merge is complete**
- The workflow runs automatically whenever Claude is mentioned in PR or
issue comments
- Claude gets access to the entire PR or issue context including files,
diffs, and previous comments

### Security

- Our Anthropic API key is securely stored as a GitHub Actions secret
- Only users with write access to the repository can trigger the
workflow
- All Claude runs are stored in the GitHub Actions run history
- Claude's default tools are limited to reading/writing files and
interacting with our repo by creating comments, branches, and commits.
- We can add more allowed tools by adding them to the workflow file
like:

```
allowed_tools: Bash(npm install),Bash(npm run build),Bash(npm run lint),Bash(npm run test)
```

There's more information in the [Claude Code action
repo](https://github.com/anthropics/claude-code-action).

After merging this PR, let's try mentioning @claude in a comment on any
PR to get started!





<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Adds GitHub Actions to integrate Claude Code for automated PR reviews
and comment-triggered help. Enables @claude to review code and perform
tasks using repository context.

- **New Features**
- claude.yml: Runs when @claude is mentioned in issue/PR comments or
reviews, or in issue title/body; uses anthropics/claude-code-action@v1
with actions: read to access CI; requires secrets.ANTHROPIC_API_KEY.
- claude-code-review.yml: Auto-reviews PRs on open/sync for quality,
bugs, performance, security, and tests; posts feedback via gh; uses
claude.md for guidance; includes optional filters and limited allowed
tools.

- **Migration**
  - Add ANTHROPIC_API_KEY to repository secrets.
  - Merge this PR, then mention @claude in a PR or issue to trigger.
- Optional: adjust file path filters, author filters, or allowed tools.

<sup>Written for commit d7e4303cb35e95db44026974a055d3d4f9fd50c4.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->

---------

Co-authored-by: Miguel <36487034+miguelg719@users.noreply.github.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-12-04 12:18:29 -08:00
Dominic Saadi
c7ee968011
Try upgrading NPM for Trusted Publishing (#1339)
# why


# what changed


# test plan


<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Enable npm Trusted Publishing in the release workflow by upgrading to
setup-node v4 and installing npm 11.5+ in CI. This should allow
tokenless canary publishes from CI.

## Why:
- Trusted Publishing requires npm >= 11.5.1.
- Our release job used setup-node v3 and an older npm.

## What:
- Upgrade actions/setup-node from v3 to v4 in release.yml.
- Add step to install latest npm globally (ensures >= 11.5.1).

## Test Plan:
- [ ] Run the canary release workflow on this branch.
- [ ] Verify npm -v in logs shows >= 11.5.1.
- [ ] Confirm publish succeeds without NPM_TOKEN (uses OIDC Trusted
Publishing).
- [ ] Check the canary publish on npm (version and tag).

<sup>Written for commit 7ab34e4f03d0f6b1280a39cb3c44d897255830f6.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-12-02 17:55:27 -08:00
Miguel
a76cc61763
Canary publish fix 2 (#1338)
# why

# what changed

# test plan


<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Simplified the canary publish step by removing manual pnpm auth setup in
the release workflow. The job now relies on NODE_AUTH_TOKEN from the
environment.

## Why:
- The pnpm auth config was redundant and could add noisy logs.
- Publishing already uses NODE_AUTH_TOKEN via env.

## What:
- Removed pnpm config set
//registry.npmjs.org/:_authToken=${NODE_AUTH_TOKEN} from the Publish
Canary step.

## Test Plan:
- [ ] Trigger the release workflow on main.
- [ ] Verify canary publish completes successfully.
- [ ] Confirm auth is picked up from NODE_AUTH_TOKEN.
- [ ] Ensure no token value appears in logs.

<sup>Written for commit d0908dbeb0fa013ba23368be171365d5ecb98bab.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-12-02 17:43:01 -08:00
Miguel
cfa9c1aa37
Fix release workflow (#1337)
# why

# what changed

# test plan


<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes the release workflow by enabling OIDC (id-token) permissions and
removing the extra npm auth env. This unblocks publish and canary runs
on main.

## Why:
- Release jobs were failing due to missing OIDC permission and
conflicting NODE_AUTH_TOKEN.
- We only need GITHUB_TOKEN and id-token for our publish steps.

## What:
- Add id-token: write to workflow permissions.
- Remove NODE_AUTH_TOKEN from publish and canary steps.
- Keep GITHUB_TOKEN for GitHub operations.

## Test Plan:
- [ ] Push to main and confirm “Publish Canary” job completes.
- [ ] Create a release/tag and confirm “Publish” job completes with no
auth errors.
- [ ] Verify logs show an OIDC token request (id-token) during publish.
- [ ] Confirm the package or canary release appears in the
registry/releases.

<sup>Written for commit 18a81b8dd6b8c97dd7be11999ac8c1b6f0069559.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-12-02 17:33:29 -08:00
Sean McGuire
4082b65e23
[chore]: fix e2e local tests in CI (#1323)
# why
- our `e2e:local` tests now require LLM API keys
- these are currently unavailable during the `e2e:local` step in CI,
which causes them to fail
# what changed
- added the relevant LLM API keys to the `env` in CI














<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes failing e2e:local tests in CI by skipping forks and passing the
required API keys to the job.

# Why:
- E2E local tests were failing due to missing secrets and forked PRs not
having access to them.

# What:
- Added job condition to run only on pushes and same-repo PRs (skip
forks).
- Passed required env secrets: OPENAI_API_KEY, ANTHROPIC_API_KEY,
GOOGLE_GENERATIVE_AI_API_KEY, BROWSERBASE_API_KEY,
BROWSERBASE_PROJECT_ID.
- Added ANTHROPIC_API_KEY to turbo.json env for the e2e:local task.

# Test Plan:
- Push a branch in this repo: e2e:local job runs and passes.
- Open a PR from this repo: job runs and external API calls succeed.
- Open a PR from a fork: e2e:local job is skipped.
- Run tests locally with required API keys set: tests pass.

<sup>Written for commit 7eb1ed9f45f51ee59e56266071d50e3f4c1d8a82.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-12-01 11:12:15 -08:00
tkattkat
86975e795d
Agents streaming (#1315)
# why

Simplify the agent streaming API by moving the `stream` flag into agent
configuration. This allows TypeScript to properly infer return types at
compile time and provides a cleaner API with a single `execute()`
method.

# what changed

- Added `stream?: boolean` to `AgentConfig`
- `agent({ stream: true }).execute()` → returns `AgentStreamResult`
- `agent().execute()` → returns `AgentResult` (unchanged)
- Added TypeScript overloads for proper type inference
- Added streaming support to `V3AgentHandler` and caching layer
- Extracted shared setup logic into `prepareAgentExecution()` helper

# test plan

- tested locally

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Sean McGuire <seanmcguire1@outlook.com>
2025-11-26 15:51:25 -08:00
Sean McGuire
6cb41bf82f
[CI] install stable chrome (#1293)
# why
- chrome launcher was installing canary, which broke some unit tests
# what changed
- added install step for a stable chrome version in `ci.yml`
- also lowered worker count for `shadow-frame.spec.ts`
2025-11-20 13:03:01 -08:00
Sean McGuire
04d68f9734
rm pnpm version specifier from CI (#1286)
# why

# what changed

# test plan
2025-11-18 16:24:27 -08:00
Filip Michalsky
44bb4f51dc
Fm/stg 956 make ci faster (#1246)
# why

CI was slow due to sequential eval jobs, repeated builds, and
inefficient dependency management.

# what changed

Parallel Execution:
- Eval jobs now run in parallel instead of sequentially (regression →
all others in parallel)
  - Reduces total CI time from ~25 mins to ~12 mins

  Build Caching:
  - Build artifacts uploaded once and reused by all eval jobs
  - Eliminates 6 redundant builds (~2-3 min savings each)

  Smart Skipping:
  - skip-evals label: skip all evals
  - skip-regression-evals label: skip regression only
  - Auto-skip for docs-only changes (not on main)

  Dependency Optimization:
  - Use pnpm/action-setup@v4 with caching
  - Use --frozen-lockfile for deterministic installs
  - Remove unnecessary node_modules cleanup

  Test Parallelization:
  - Local tests: 2→3 workers (Browserbase), 2→4 workers (standard)
  - Better CI vs local environment detection

# test plan

 - Verify build artifacts upload/download correctly
  - Confirm eval jobs run in parallel after regression
  - Test skip-evals and skip-regression-evals labels work
  - Verify docs-only changes skip evals (except on main)
  - Measure CI time reduction (~50-60% expected)
  - Confirm no test flakiness from increased parallelization
  - All tests, lint, and build pass

---------

Co-authored-by: Sean McGuire <75873287+seanmcguire12@users.noreply.github.com>
2025-11-17 14:22:18 -05:00
Sam F
ef38ccf0a2
Add minimal contract test for public api (#1255)
# why
We want to be able to see what the public API shape looks like - which
top level functions are exported.

# what changed
Added vitest and a minimal test file that checks which functions are
exported at the root level. If new functions are exported or removed,
this test will catch that. I also added one example of drilling down and
testing the parameters of StagehandMetrics — in the future we might want
to do this for all exports.

Curious if this surfaces and exports that don't need to be exported or
not.

# test plan
Just added tests. They run in the CI pipeline or can be run with `pnpm
--filter @browserbasehq/stagehand run test:vitest`
2025-11-17 11:00:31 -08:00
Miguel
7f36683178
remove sonnet 3.5 update with haiku (#1188)
# why

# what changed

# test plan
2025-10-28 14:06:51 -07:00
Sean McGuire
4633f4c7a2
dont install playwright browsers in CI (#1186)
# why

# what changed

# test plan
2025-10-28 08:20:13 -07:00
Filip Michalsky
3c1c7cd2b5
first ci attempt for v3 (#1169)
# why
v3 routes were previously not on CI

# what changed
added CI runners for v3

# test plan
CI passes
2025-10-27 21:16:50 -07:00
Miguel
6e57b8e226
Update issue templates (#1035)
To help debug issues faster, and let users propose features
2025-08-28 10:34:21 -07:00
Filip Michalsky
09b5e1e9c2
work on adding voyager to evals (#959)
⏺ ## Why
Add WebVoyager and GAIA evaluation suites to benchmark Stagehand's web
navigation and
   reasoning capabilities against industry-standard datasets.

  ## What Changed
- Added WebVoyager eval suite with 643 test cases for web navigation
tasks
- Added GAIA eval suite with 90 test cases for general AI assistant
tasks
  - Refactored eval infrastructure to support sampling and filtering
- Created reusable utilities for JSONL parsing and test case generation
  - Added configuration for new eval suites in `evals.config.json`

  ### Environment Variables
  - `EVAL_WEBVOYAGER_SAMPLE`: Random sample size from WebVoyager dataset
  - `EVAL_WEBVOYAGER_LIMIT`: Max cases to run (default: 25)
  - `EVAL_GAIA_SAMPLE`: Random sample size from GAIA dataset
  - `EVAL_GAIA_LIMIT`: Max cases to run (default: 25)
  - `EVAL_GAIA_LEVEL`: Filter GAIA by difficulty level (1, 2, or 3)

  ### Sampling Strategy
The sampling implementation uses Fisher-Yates shuffle for unbiased
random selection
when `SAMPLE` is specified, otherwise takes the first `LIMIT` cases.
This allows for
  both deterministic (first N) and randomized (sample N) test runs.

  ## Test Plan
  ```bash
  # Test WebVoyager with OpenAI
  EVAL_SUITE=webvoyager EVAL_WEBVOYAGER_SAMPLE=1
  EVAL_MODEL=openai/gpt-4o-computer-use-preview pnpm run evals

  # Test WebVoyager with Claude
  EVAL_SUITE=webvoyager EVAL_WEBVOYAGER_SAMPLE=1
  EVAL_MODEL=anthropic/claude-3-5-sonnet-20241022 pnpm run evals

  # Test GAIA with OpenAI
EVAL_SUITE=gaia EVAL_GAIA_SAMPLE=1
EVAL_MODEL=openai/gpt-4o-computer-use-preview pnpm
   run evals

  # Test GAIA with Claude
EVAL_SUITE=gaia EVAL_GAIA_SAMPLE=1
EVAL_MODEL=anthropic/claude-3-5-sonnet-20241022
  pnpm run evals

  # Verify existing evals still work
  pnpm run evals
  ```

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Miguel <36487034+miguelg719@users.noreply.github.com>
2025-08-27 16:40:53 -04:00
Miguel
5d668b8a6b
fix parity workflow (#1017)
# why

# what changed

# test plan
2025-08-26 12:31:46 -07:00
Miguel
f75f5fb848
consolidate llm provider for evals (#893)
# why

# what changed

# test plan
2025-07-17 15:18:04 -07:00
Miguel
a5a0107a22
add feature parity workflow (#853)
# why
To keep up with changes on client SDKs

# what changed
Added a workflow 'Feature Parity' that, given an added (specific) label
to a PR, will open an issue on the sibling repo describing the PR for
future implementation

# test plan

---------

Co-authored-by: Roaring <216452114+the-roaring@users.noreply.github.com>
2025-06-30 15:09:08 -07:00