2024-03-20 10:09:26 -07:00
|
|
|
{
|
2025-10-10 12:29:49 -07:00
|
|
|
"name": "stagehand-workspace",
|
|
|
|
|
"version": "0.0.0",
|
|
|
|
|
"private": true,
|
|
|
|
|
"description": "Stagehand monorepo workspace",
|
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632)
# why
- unit tests, e2e integration tests, and evals were taking 10~15 minutes
per pr
- finding the specific failing eval or test was tedious and required
reading tons of log output
- we weren't leveraging pnpm or turbo caches or github action's native
chromium install
- tests were flaky because CDP would randomly fail to connect when
github actions runner was overloaded
- e2e bb tests were not actually running against bb, so it was
previously just double-running e2e:local and reporting it as bb
- tsup has been deprecated for a while and should be replaced by esbuild
# what changed
- parallelized all CI tests and broke out evals and tests into
individual matrix jobs
- created unified node+pnpm+turbo setup action that all other actions
call
- verified vanilla chromium is able to launch inside all the e2e local
jobs before running any of our own code to catch resource contention
flakyness
- add proper esm and csj builds with sourcemaps and types for aligned
coverage
- remove tsup dependency
- switch to new evals cli
- add coverage reporting
- fix some flaky tests needed to get everything passing reliably
<img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM"
src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7"
/>
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Speeds up CI and makes tests stable by running Vitest/Playwright/Evals
on the built ESM dist with cached artifacts, verified runner Chromium,
weighted Browserbase routing, and merged coverage. Publishes a unified
ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes
Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test
reporting, addressing Linear STG‑1232.
- **New Features**
- CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium
launch check, region selection, CTRF/V8 uploads); reuse artifacts; use
Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports;
V8 coverage normalization/merge with cancellation safety; push triggers
enabled.
- Tests: run from dist/esm; env reporter; true Browserbase runs with
weighted/randomized regions; hard timeouts for LLM and session
create/CDP connect; faster multi‑click dispatch; higher timeouts and
targeted skips; cleanup helpers; fixes for init‑script,
wait‑for‑selector, and screenshot reliability.
- Build/SDK/SEA: export map (ESM import + CJS require) with
sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain
logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and
exported; getAISDKLanguageModel added; replay metrics now include
method/parameters/result/timestamps/cost.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV
(replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS,
BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS;
optionally set BROWSERBASE_REGION_DISTRIBUTION.
- Remove dotenv; set env vars directly and import
@browserbasehq/stagehand via the export map (avoid dist paths).
<sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types.
CI runs Vitest/Playwright and the new Evals CLI against the built ESM
dist using the runner’s Chromium (launch‑verified), with region‑weighted
Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and
more stable tests (timeouts, Browserbase skips, understudy click/API
fixes).
Speed up CI by running Vitest and Playwright tests in parallel matrices,
add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard
timeouts and better cleanup. Server integration tests now use the
runner’s Chromium for reliability and are folded into the main CI.
- **Refactors**
- Parallelized core unit tests via discover-core-tests/core-unit-tests
(Vitest matrix, fail-fast).
- Parallelized E2E via discover-e2e-tests (Playwright local +
Browserbase) gated on core changes; weighted Browserbase region
selection; verify Chromium/CDP before tests; local uses runner Chromium
via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when
CTRF_JUNIT_PATH is set.
- Unified Lint & Build with Turbo caching and shared artifacts;
tests/evals download and reuse; prepare script skips build in CI; SEA
builds support sourcemaps.
- Enforced configurable timeouts and parallelism via env (LLM_MAX_MS,
BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS,
LOCAL_SESSION_LIMIT_PER_E2E_TEST,
BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and
Browserbase hard timeouts; improved click event dispatch to reduce
remote flakiness.
- Explicitly close Browserbase sessions in tests and evals to free
capacity.
- Added composite actions: setup-node-pnpm-turbo,
verify-chromium-launch, select-browserbase-region, publish-ctrf-report
(JUnit→CTRF and evals), upload-v8-coverage.
- Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter
for CI logs; removed the standalone stagehand-server tests workflow and
ran server tests in the unified CI with Node’s test runner and GitHub
Actions Chromium.
- **New Features**
- Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map;
types from ESM; SEA binaries built from dist with sourcemaps; server SEA
job targets CJS; new build/test/coverage scripts; Node loader maps
package imports to dist/esm; ESM‑safe pathing (fileURLToPath);
ESLint/Prettier tweaks; dotenv removed.
- CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch,
select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses
runner Chromium via CHROME_PATH with --no-sandbox and launch checks;
weighted Browserbase region selection; unique CTRF artifacts; V8
coverage normalized/merged with debug logs and graceful cancellation;
Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled.
- Tests & API: Vitest/Playwright run on dist/esm; new env reporter and
closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and
targeted skips (e.g., console on Browserbase) reduce flakiness; local
launches honor CHROME_PATH; replay metrics now include
parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI
clients moved to lib/v3/external_clients and exported; env‑driven hard
timeouts for LLM ops and Browserbase session create/CDP connect;
understudy API usage fixes.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune
LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and
BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set
BROWSERBASE_REGION_DISTRIBUTION.
- dotenv support is removed; set env vars explicitly in dev and CI.
- Import @browserbasehq/stagehand via the export map; do not reference
dist paths or prior tsup outputs.
<sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
|
|
|
"type": "module",
|
2024-05-14 14:26:59 -07:00
|
|
|
"scripts": {
|
2025-10-10 12:29:49 -07:00
|
|
|
"build": "turbo run build",
|
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632)
# why
- unit tests, e2e integration tests, and evals were taking 10~15 minutes
per pr
- finding the specific failing eval or test was tedious and required
reading tons of log output
- we weren't leveraging pnpm or turbo caches or github action's native
chromium install
- tests were flaky because CDP would randomly fail to connect when
github actions runner was overloaded
- e2e bb tests were not actually running against bb, so it was
previously just double-running e2e:local and reporting it as bb
- tsup has been deprecated for a while and should be replaced by esbuild
# what changed
- parallelized all CI tests and broke out evals and tests into
individual matrix jobs
- created unified node+pnpm+turbo setup action that all other actions
call
- verified vanilla chromium is able to launch inside all the e2e local
jobs before running any of our own code to catch resource contention
flakyness
- add proper esm and csj builds with sourcemaps and types for aligned
coverage
- remove tsup dependency
- switch to new evals cli
- add coverage reporting
- fix some flaky tests needed to get everything passing reliably
<img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM"
src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7"
/>
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Speeds up CI and makes tests stable by running Vitest/Playwright/Evals
on the built ESM dist with cached artifacts, verified runner Chromium,
weighted Browserbase routing, and merged coverage. Publishes a unified
ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes
Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test
reporting, addressing Linear STG‑1232.
- **New Features**
- CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium
launch check, region selection, CTRF/V8 uploads); reuse artifacts; use
Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports;
V8 coverage normalization/merge with cancellation safety; push triggers
enabled.
- Tests: run from dist/esm; env reporter; true Browserbase runs with
weighted/randomized regions; hard timeouts for LLM and session
create/CDP connect; faster multi‑click dispatch; higher timeouts and
targeted skips; cleanup helpers; fixes for init‑script,
wait‑for‑selector, and screenshot reliability.
- Build/SDK/SEA: export map (ESM import + CJS require) with
sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain
logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and
exported; getAISDKLanguageModel added; replay metrics now include
method/parameters/result/timestamps/cost.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV
(replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS,
BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS;
optionally set BROWSERBASE_REGION_DISTRIBUTION.
- Remove dotenv; set env vars directly and import
@browserbasehq/stagehand via the export map (avoid dist paths).
<sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types.
CI runs Vitest/Playwright and the new Evals CLI against the built ESM
dist using the runner’s Chromium (launch‑verified), with region‑weighted
Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and
more stable tests (timeouts, Browserbase skips, understudy click/API
fixes).
Speed up CI by running Vitest and Playwright tests in parallel matrices,
add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard
timeouts and better cleanup. Server integration tests now use the
runner’s Chromium for reliability and are folded into the main CI.
- **Refactors**
- Parallelized core unit tests via discover-core-tests/core-unit-tests
(Vitest matrix, fail-fast).
- Parallelized E2E via discover-e2e-tests (Playwright local +
Browserbase) gated on core changes; weighted Browserbase region
selection; verify Chromium/CDP before tests; local uses runner Chromium
via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when
CTRF_JUNIT_PATH is set.
- Unified Lint & Build with Turbo caching and shared artifacts;
tests/evals download and reuse; prepare script skips build in CI; SEA
builds support sourcemaps.
- Enforced configurable timeouts and parallelism via env (LLM_MAX_MS,
BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS,
LOCAL_SESSION_LIMIT_PER_E2E_TEST,
BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and
Browserbase hard timeouts; improved click event dispatch to reduce
remote flakiness.
- Explicitly close Browserbase sessions in tests and evals to free
capacity.
- Added composite actions: setup-node-pnpm-turbo,
verify-chromium-launch, select-browserbase-region, publish-ctrf-report
(JUnit→CTRF and evals), upload-v8-coverage.
- Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter
for CI logs; removed the standalone stagehand-server tests workflow and
ran server tests in the unified CI with Node’s test runner and GitHub
Actions Chromium.
- **New Features**
- Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map;
types from ESM; SEA binaries built from dist with sourcemaps; server SEA
job targets CJS; new build/test/coverage scripts; Node loader maps
package imports to dist/esm; ESM‑safe pathing (fileURLToPath);
ESLint/Prettier tweaks; dotenv removed.
- CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch,
select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses
runner Chromium via CHROME_PATH with --no-sandbox and launch checks;
weighted Browserbase region selection; unique CTRF artifacts; V8
coverage normalized/merged with debug logs and graceful cancellation;
Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled.
- Tests & API: Vitest/Playwright run on dist/esm; new env reporter and
closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and
targeted skips (e.g., console on Browserbase) reduce flakiness; local
launches honor CHROME_PATH; replay metrics now include
parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI
clients moved to lib/v3/external_clients and exported; env‑driven hard
timeouts for LLM ops and Browserbase session create/CDP connect;
understudy API usage fixes.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune
LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and
BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set
BROWSERBASE_REGION_DISTRIBUTION.
- dotenv support is removed; set env vars explicitly in dev and CI.
- Import @browserbasehq/stagehand via the export map; do not reference
dist paths or prior tsup outputs.
<sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
|
|
|
"build:full": "turbo run build",
|
2026-02-19 15:35:37 -08:00
|
|
|
"build:cjs": "turbo run build:cjs",
|
2025-10-10 12:29:49 -07:00
|
|
|
"build:cli": "turbo run build:cli",
|
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632)
# why
- unit tests, e2e integration tests, and evals were taking 10~15 minutes
per pr
- finding the specific failing eval or test was tedious and required
reading tons of log output
- we weren't leveraging pnpm or turbo caches or github action's native
chromium install
- tests were flaky because CDP would randomly fail to connect when
github actions runner was overloaded
- e2e bb tests were not actually running against bb, so it was
previously just double-running e2e:local and reporting it as bb
- tsup has been deprecated for a while and should be replaced by esbuild
# what changed
- parallelized all CI tests and broke out evals and tests into
individual matrix jobs
- created unified node+pnpm+turbo setup action that all other actions
call
- verified vanilla chromium is able to launch inside all the e2e local
jobs before running any of our own code to catch resource contention
flakyness
- add proper esm and csj builds with sourcemaps and types for aligned
coverage
- remove tsup dependency
- switch to new evals cli
- add coverage reporting
- fix some flaky tests needed to get everything passing reliably
<img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM"
src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7"
/>
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Speeds up CI and makes tests stable by running Vitest/Playwright/Evals
on the built ESM dist with cached artifacts, verified runner Chromium,
weighted Browserbase routing, and merged coverage. Publishes a unified
ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes
Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test
reporting, addressing Linear STG‑1232.
- **New Features**
- CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium
launch check, region selection, CTRF/V8 uploads); reuse artifacts; use
Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports;
V8 coverage normalization/merge with cancellation safety; push triggers
enabled.
- Tests: run from dist/esm; env reporter; true Browserbase runs with
weighted/randomized regions; hard timeouts for LLM and session
create/CDP connect; faster multi‑click dispatch; higher timeouts and
targeted skips; cleanup helpers; fixes for init‑script,
wait‑for‑selector, and screenshot reliability.
- Build/SDK/SEA: export map (ESM import + CJS require) with
sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain
logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and
exported; getAISDKLanguageModel added; replay metrics now include
method/parameters/result/timestamps/cost.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV
(replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS,
BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS;
optionally set BROWSERBASE_REGION_DISTRIBUTION.
- Remove dotenv; set env vars directly and import
@browserbasehq/stagehand via the export map (avoid dist paths).
<sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types.
CI runs Vitest/Playwright and the new Evals CLI against the built ESM
dist using the runner’s Chromium (launch‑verified), with region‑weighted
Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and
more stable tests (timeouts, Browserbase skips, understudy click/API
fixes).
Speed up CI by running Vitest and Playwright tests in parallel matrices,
add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard
timeouts and better cleanup. Server integration tests now use the
runner’s Chromium for reliability and are folded into the main CI.
- **Refactors**
- Parallelized core unit tests via discover-core-tests/core-unit-tests
(Vitest matrix, fail-fast).
- Parallelized E2E via discover-e2e-tests (Playwright local +
Browserbase) gated on core changes; weighted Browserbase region
selection; verify Chromium/CDP before tests; local uses runner Chromium
via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when
CTRF_JUNIT_PATH is set.
- Unified Lint & Build with Turbo caching and shared artifacts;
tests/evals download and reuse; prepare script skips build in CI; SEA
builds support sourcemaps.
- Enforced configurable timeouts and parallelism via env (LLM_MAX_MS,
BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS,
LOCAL_SESSION_LIMIT_PER_E2E_TEST,
BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and
Browserbase hard timeouts; improved click event dispatch to reduce
remote flakiness.
- Explicitly close Browserbase sessions in tests and evals to free
capacity.
- Added composite actions: setup-node-pnpm-turbo,
verify-chromium-launch, select-browserbase-region, publish-ctrf-report
(JUnit→CTRF and evals), upload-v8-coverage.
- Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter
for CI logs; removed the standalone stagehand-server tests workflow and
ran server tests in the unified CI with Node’s test runner and GitHub
Actions Chromium.
- **New Features**
- Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map;
types from ESM; SEA binaries built from dist with sourcemaps; server SEA
job targets CJS; new build/test/coverage scripts; Node loader maps
package imports to dist/esm; ESM‑safe pathing (fileURLToPath);
ESLint/Prettier tweaks; dotenv removed.
- CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch,
select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses
runner Chromium via CHROME_PATH with --no-sandbox and launch checks;
weighted Browserbase region selection; unique CTRF artifacts; V8
coverage normalized/merged with debug logs and graceful cancellation;
Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled.
- Tests & API: Vitest/Playwright run on dist/esm; new env reporter and
closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and
targeted skips (e.g., console on Browserbase) reduce flakiness; local
launches honor CHROME_PATH; replay metrics now include
parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI
clients moved to lib/v3/external_clients and exported; env‑driven hard
timeouts for LLM ops and Browserbase session create/CDP connect;
understudy API usage fixes.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune
LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and
BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set
BROWSERBASE_REGION_DISTRIBUTION.
- dotenv support is removed; set env vars explicitly in dev and CI.
- Import @browserbasehq/stagehand via the export map; do not reference
dist paths or prior tsup outputs.
<sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
|
|
|
"build:esm": "turbo run build:esm",
|
|
|
|
|
"build:sea": "turbo run build:sea:esm",
|
|
|
|
|
"build:sea:esm": "turbo run build:sea:esm",
|
|
|
|
|
"build:sea:cjs": "turbo run build:sea:cjs",
|
2025-10-10 12:29:49 -07:00
|
|
|
"lint": "turbo run lint",
|
2025-12-15 15:05:23 -05:00
|
|
|
"format": "prettier --write .",
|
2025-10-10 12:29:49 -07:00
|
|
|
"prettier": "prettier --write .",
|
2024-12-08 17:22:31 -08:00
|
|
|
"eslint": "eslint .",
|
2026-03-12 17:44:59 -07:00
|
|
|
"test": "turbo run test:core test:e2e test:server test:evals test:cli",
|
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632)
# why
- unit tests, e2e integration tests, and evals were taking 10~15 minutes
per pr
- finding the specific failing eval or test was tedious and required
reading tons of log output
- we weren't leveraging pnpm or turbo caches or github action's native
chromium install
- tests were flaky because CDP would randomly fail to connect when
github actions runner was overloaded
- e2e bb tests were not actually running against bb, so it was
previously just double-running e2e:local and reporting it as bb
- tsup has been deprecated for a while and should be replaced by esbuild
# what changed
- parallelized all CI tests and broke out evals and tests into
individual matrix jobs
- created unified node+pnpm+turbo setup action that all other actions
call
- verified vanilla chromium is able to launch inside all the e2e local
jobs before running any of our own code to catch resource contention
flakyness
- add proper esm and csj builds with sourcemaps and types for aligned
coverage
- remove tsup dependency
- switch to new evals cli
- add coverage reporting
- fix some flaky tests needed to get everything passing reliably
<img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM"
src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7"
/>
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Speeds up CI and makes tests stable by running Vitest/Playwright/Evals
on the built ESM dist with cached artifacts, verified runner Chromium,
weighted Browserbase routing, and merged coverage. Publishes a unified
ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes
Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test
reporting, addressing Linear STG‑1232.
- **New Features**
- CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium
launch check, region selection, CTRF/V8 uploads); reuse artifacts; use
Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports;
V8 coverage normalization/merge with cancellation safety; push triggers
enabled.
- Tests: run from dist/esm; env reporter; true Browserbase runs with
weighted/randomized regions; hard timeouts for LLM and session
create/CDP connect; faster multi‑click dispatch; higher timeouts and
targeted skips; cleanup helpers; fixes for init‑script,
wait‑for‑selector, and screenshot reliability.
- Build/SDK/SEA: export map (ESM import + CJS require) with
sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain
logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and
exported; getAISDKLanguageModel added; replay metrics now include
method/parameters/result/timestamps/cost.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV
(replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS,
BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS;
optionally set BROWSERBASE_REGION_DISTRIBUTION.
- Remove dotenv; set env vars directly and import
@browserbasehq/stagehand via the export map (avoid dist paths).
<sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types.
CI runs Vitest/Playwright and the new Evals CLI against the built ESM
dist using the runner’s Chromium (launch‑verified), with region‑weighted
Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and
more stable tests (timeouts, Browserbase skips, understudy click/API
fixes).
Speed up CI by running Vitest and Playwright tests in parallel matrices,
add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard
timeouts and better cleanup. Server integration tests now use the
runner’s Chromium for reliability and are folded into the main CI.
- **Refactors**
- Parallelized core unit tests via discover-core-tests/core-unit-tests
(Vitest matrix, fail-fast).
- Parallelized E2E via discover-e2e-tests (Playwright local +
Browserbase) gated on core changes; weighted Browserbase region
selection; verify Chromium/CDP before tests; local uses runner Chromium
via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when
CTRF_JUNIT_PATH is set.
- Unified Lint & Build with Turbo caching and shared artifacts;
tests/evals download and reuse; prepare script skips build in CI; SEA
builds support sourcemaps.
- Enforced configurable timeouts and parallelism via env (LLM_MAX_MS,
BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS,
LOCAL_SESSION_LIMIT_PER_E2E_TEST,
BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and
Browserbase hard timeouts; improved click event dispatch to reduce
remote flakiness.
- Explicitly close Browserbase sessions in tests and evals to free
capacity.
- Added composite actions: setup-node-pnpm-turbo,
verify-chromium-launch, select-browserbase-region, publish-ctrf-report
(JUnit→CTRF and evals), upload-v8-coverage.
- Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter
for CI logs; removed the standalone stagehand-server tests workflow and
ran server tests in the unified CI with Node’s test runner and GitHub
Actions Chromium.
- **New Features**
- Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map;
types from ESM; SEA binaries built from dist with sourcemaps; server SEA
job targets CJS; new build/test/coverage scripts; Node loader maps
package imports to dist/esm; ESM‑safe pathing (fileURLToPath);
ESLint/Prettier tweaks; dotenv removed.
- CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch,
select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses
runner Chromium via CHROME_PATH with --no-sandbox and launch checks;
weighted Browserbase region selection; unique CTRF artifacts; V8
coverage normalized/merged with debug logs and graceful cancellation;
Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled.
- Tests & API: Vitest/Playwright run on dist/esm; new env reporter and
closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and
targeted skips (e.g., console on Browserbase) reduce flakiness; local
launches honor CHROME_PATH; replay metrics now include
parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI
clients moved to lib/v3/external_clients and exported; env‑driven hard
timeouts for LLM ops and Browserbase session create/CDP connect;
understudy API usage fixes.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune
LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and
BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set
BROWSERBASE_REGION_DISTRIBUTION.
- dotenv support is removed; set env vars explicitly in dev and CI.
- Import @browserbasehq/stagehand via the export map; do not reference
dist paths or prior tsup outputs.
<sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
|
|
|
"test:core": "turbo run test:core --",
|
|
|
|
|
"test:core:local": "STAGEHAND_BROWSER_TARGET=local pnpm run test:core --",
|
|
|
|
|
"test:core:bb": "STAGEHAND_BROWSER_TARGET=browserbase pnpm run test:core --",
|
|
|
|
|
"test:e2e": "turbo run test:e2e --",
|
|
|
|
|
"test:e2e:local": "STAGEHAND_BROWSER_TARGET=local pnpm run test:e2e --",
|
|
|
|
|
"test:e2e:bb": "STAGEHAND_BROWSER_TARGET=browserbase pnpm run test:e2e --",
|
|
|
|
|
"test:server": "turbo run test:server --",
|
|
|
|
|
"test:server:sea": "STAGEHAND_SERVER_TARGET=sea pnpm run test:server --",
|
|
|
|
|
"test:server:local": "STAGEHAND_SERVER_TARGET=local pnpm run test:server --",
|
|
|
|
|
"test:server:remote": "STAGEHAND_SERVER_TARGET=remote pnpm run test:server --",
|
|
|
|
|
"test:evals": "turbo run test:evals --",
|
|
|
|
|
"test:evals:local": "STAGEHAND_BROWSER_TARGET=local pnpm run test:evals --",
|
|
|
|
|
"test:evals:bb": "STAGEHAND_BROWSER_TARGET=browserbase pnpm run test:evals --",
|
|
|
|
|
"coverage:merge": "pnpm -w exec tsx packages/core/scripts/coverage.ts merge",
|
2025-10-10 12:29:49 -07:00
|
|
|
"docs": "turbo run docs",
|
|
|
|
|
"dev": "turbo run dev",
|
2025-10-29 12:16:22 -07:00
|
|
|
"example": "pnpm --filter @browserbasehq/stagehand run example --",
|
2025-10-10 12:29:49 -07:00
|
|
|
"cache:clear": "turbo run build --force",
|
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632)
# why
- unit tests, e2e integration tests, and evals were taking 10~15 minutes
per pr
- finding the specific failing eval or test was tedious and required
reading tons of log output
- we weren't leveraging pnpm or turbo caches or github action's native
chromium install
- tests were flaky because CDP would randomly fail to connect when
github actions runner was overloaded
- e2e bb tests were not actually running against bb, so it was
previously just double-running e2e:local and reporting it as bb
- tsup has been deprecated for a while and should be replaced by esbuild
# what changed
- parallelized all CI tests and broke out evals and tests into
individual matrix jobs
- created unified node+pnpm+turbo setup action that all other actions
call
- verified vanilla chromium is able to launch inside all the e2e local
jobs before running any of our own code to catch resource contention
flakyness
- add proper esm and csj builds with sourcemaps and types for aligned
coverage
- remove tsup dependency
- switch to new evals cli
- add coverage reporting
- fix some flaky tests needed to get everything passing reliably
<img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM"
src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7"
/>
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Speeds up CI and makes tests stable by running Vitest/Playwright/Evals
on the built ESM dist with cached artifacts, verified runner Chromium,
weighted Browserbase routing, and merged coverage. Publishes a unified
ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes
Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test
reporting, addressing Linear STG‑1232.
- **New Features**
- CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium
launch check, region selection, CTRF/V8 uploads); reuse artifacts; use
Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports;
V8 coverage normalization/merge with cancellation safety; push triggers
enabled.
- Tests: run from dist/esm; env reporter; true Browserbase runs with
weighted/randomized regions; hard timeouts for LLM and session
create/CDP connect; faster multi‑click dispatch; higher timeouts and
targeted skips; cleanup helpers; fixes for init‑script,
wait‑for‑selector, and screenshot reliability.
- Build/SDK/SEA: export map (ESM import + CJS require) with
sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain
logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and
exported; getAISDKLanguageModel added; replay metrics now include
method/parameters/result/timestamps/cost.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV
(replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS,
BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS;
optionally set BROWSERBASE_REGION_DISTRIBUTION.
- Remove dotenv; set env vars directly and import
@browserbasehq/stagehand via the export map (avoid dist paths).
<sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types.
CI runs Vitest/Playwright and the new Evals CLI against the built ESM
dist using the runner’s Chromium (launch‑verified), with region‑weighted
Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and
more stable tests (timeouts, Browserbase skips, understudy click/API
fixes).
Speed up CI by running Vitest and Playwright tests in parallel matrices,
add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard
timeouts and better cleanup. Server integration tests now use the
runner’s Chromium for reliability and are folded into the main CI.
- **Refactors**
- Parallelized core unit tests via discover-core-tests/core-unit-tests
(Vitest matrix, fail-fast).
- Parallelized E2E via discover-e2e-tests (Playwright local +
Browserbase) gated on core changes; weighted Browserbase region
selection; verify Chromium/CDP before tests; local uses runner Chromium
via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when
CTRF_JUNIT_PATH is set.
- Unified Lint & Build with Turbo caching and shared artifacts;
tests/evals download and reuse; prepare script skips build in CI; SEA
builds support sourcemaps.
- Enforced configurable timeouts and parallelism via env (LLM_MAX_MS,
BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS,
LOCAL_SESSION_LIMIT_PER_E2E_TEST,
BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and
Browserbase hard timeouts; improved click event dispatch to reduce
remote flakiness.
- Explicitly close Browserbase sessions in tests and evals to free
capacity.
- Added composite actions: setup-node-pnpm-turbo,
verify-chromium-launch, select-browserbase-region, publish-ctrf-report
(JUnit→CTRF and evals), upload-v8-coverage.
- Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter
for CI logs; removed the standalone stagehand-server tests workflow and
ran server tests in the unified CI with Node’s test runner and GitHub
Actions Chromium.
- **New Features**
- Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map;
types from ESM; SEA binaries built from dist with sourcemaps; server SEA
job targets CJS; new build/test/coverage scripts; Node loader maps
package imports to dist/esm; ESM‑safe pathing (fileURLToPath);
ESLint/Prettier tweaks; dotenv removed.
- CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch,
select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses
runner Chromium via CHROME_PATH with --no-sandbox and launch checks;
weighted Browserbase region selection; unique CTRF artifacts; V8
coverage normalized/merged with debug logs and graceful cancellation;
Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled.
- Tests & API: Vitest/Playwright run on dist/esm; new env reporter and
closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and
targeted skips (e.g., console on Browserbase) reduce flakiness; local
launches honor CHROME_PATH; replay metrics now include
parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI
clients moved to lib/v3/external_clients and exported; env‑driven hard
timeouts for LLM ops and Browserbase session create/CDP connect;
understudy API usage fixes.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune
LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and
BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set
BROWSERBASE_REGION_DISTRIBUTION.
- dotenv support is removed; set env vars explicitly in dev and CI.
- Import @browserbasehq/stagehand via the export map; do not reference
dist paths or prior tsup outputs.
<sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
|
|
|
"prepare": "node packages/core/scripts/prepare.js",
|
2025-10-10 12:29:49 -07:00
|
|
|
"release": "turbo run build && changeset publish",
|
|
|
|
|
"release-canary": "turbo run build && changeset version --snapshot && changeset publish --tag alpha"
|
2024-05-14 14:26:59 -07:00
|
|
|
},
|
2025-05-03 14:41:15 -07:00
|
|
|
"devDependencies": {
|
|
|
|
|
"@changesets/changelog-github": "^0.5.0",
|
|
|
|
|
"@changesets/cli": "^2.27.9",
|
2026-02-24 14:03:33 -08:00
|
|
|
"@eslint/js": "^10.0.1",
|
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632)
# why
- unit tests, e2e integration tests, and evals were taking 10~15 minutes
per pr
- finding the specific failing eval or test was tedious and required
reading tons of log output
- we weren't leveraging pnpm or turbo caches or github action's native
chromium install
- tests were flaky because CDP would randomly fail to connect when
github actions runner was overloaded
- e2e bb tests were not actually running against bb, so it was
previously just double-running e2e:local and reporting it as bb
- tsup has been deprecated for a while and should be replaced by esbuild
# what changed
- parallelized all CI tests and broke out evals and tests into
individual matrix jobs
- created unified node+pnpm+turbo setup action that all other actions
call
- verified vanilla chromium is able to launch inside all the e2e local
jobs before running any of our own code to catch resource contention
flakyness
- add proper esm and csj builds with sourcemaps and types for aligned
coverage
- remove tsup dependency
- switch to new evals cli
- add coverage reporting
- fix some flaky tests needed to get everything passing reliably
<img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM"
src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7"
/>
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Speeds up CI and makes tests stable by running Vitest/Playwright/Evals
on the built ESM dist with cached artifacts, verified runner Chromium,
weighted Browserbase routing, and merged coverage. Publishes a unified
ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes
Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test
reporting, addressing Linear STG‑1232.
- **New Features**
- CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium
launch check, region selection, CTRF/V8 uploads); reuse artifacts; use
Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports;
V8 coverage normalization/merge with cancellation safety; push triggers
enabled.
- Tests: run from dist/esm; env reporter; true Browserbase runs with
weighted/randomized regions; hard timeouts for LLM and session
create/CDP connect; faster multi‑click dispatch; higher timeouts and
targeted skips; cleanup helpers; fixes for init‑script,
wait‑for‑selector, and screenshot reliability.
- Build/SDK/SEA: export map (ESM import + CJS require) with
sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain
logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and
exported; getAISDKLanguageModel added; replay metrics now include
method/parameters/result/timestamps/cost.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV
(replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS,
BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS;
optionally set BROWSERBASE_REGION_DISTRIBUTION.
- Remove dotenv; set env vars directly and import
@browserbasehq/stagehand via the export map (avoid dist paths).
<sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types.
CI runs Vitest/Playwright and the new Evals CLI against the built ESM
dist using the runner’s Chromium (launch‑verified), with region‑weighted
Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and
more stable tests (timeouts, Browserbase skips, understudy click/API
fixes).
Speed up CI by running Vitest and Playwright tests in parallel matrices,
add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard
timeouts and better cleanup. Server integration tests now use the
runner’s Chromium for reliability and are folded into the main CI.
- **Refactors**
- Parallelized core unit tests via discover-core-tests/core-unit-tests
(Vitest matrix, fail-fast).
- Parallelized E2E via discover-e2e-tests (Playwright local +
Browserbase) gated on core changes; weighted Browserbase region
selection; verify Chromium/CDP before tests; local uses runner Chromium
via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when
CTRF_JUNIT_PATH is set.
- Unified Lint & Build with Turbo caching and shared artifacts;
tests/evals download and reuse; prepare script skips build in CI; SEA
builds support sourcemaps.
- Enforced configurable timeouts and parallelism via env (LLM_MAX_MS,
BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS,
LOCAL_SESSION_LIMIT_PER_E2E_TEST,
BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and
Browserbase hard timeouts; improved click event dispatch to reduce
remote flakiness.
- Explicitly close Browserbase sessions in tests and evals to free
capacity.
- Added composite actions: setup-node-pnpm-turbo,
verify-chromium-launch, select-browserbase-region, publish-ctrf-report
(JUnit→CTRF and evals), upload-v8-coverage.
- Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter
for CI logs; removed the standalone stagehand-server tests workflow and
ran server tests in the unified CI with Node’s test runner and GitHub
Actions Chromium.
- **New Features**
- Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map;
types from ESM; SEA binaries built from dist with sourcemaps; server SEA
job targets CJS; new build/test/coverage scripts; Node loader maps
package imports to dist/esm; ESM‑safe pathing (fileURLToPath);
ESLint/Prettier tweaks; dotenv removed.
- CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch,
select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses
runner Chromium via CHROME_PATH with --no-sandbox and launch checks;
weighted Browserbase region selection; unique CTRF artifacts; V8
coverage normalized/merged with debug logs and graceful cancellation;
Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled.
- Tests & API: Vitest/Playwright run on dist/esm; new env reporter and
closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and
targeted skips (e.g., console on Browserbase) reduce flakiness; local
launches honor CHROME_PATH; replay metrics now include
parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI
clients moved to lib/v3/external_clients and exported; env‑driven hard
timeouts for LLM ops and Browserbase session create/CDP connect;
understudy API usage fixes.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune
LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and
BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set
BROWSERBASE_REGION_DISTRIBUTION.
- dotenv support is removed; set env vars explicitly in dev and CI.
- Import @browserbasehq/stagehand via the export map; do not reference
dist paths or prior tsup outputs.
<sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
|
|
|
"c8": "^10.1.3",
|
[chore]: add `.js` to the end of all imports (#1700)
# why
- removes the need to append .js to all imports during esm build process
- aligns us with the default standard for esm / esbuild js projects that
also need to support cjs
- this allows us to seamlessly support ESM, CJS, and TSX with no
remapping needed
# what changed
# test plan
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Standardized all internal imports and re-exports to include .js
extensions to enable native ESM resolution, while keeping CJS and TS
working without path remapping. Updated examples to use explicit
index.js entrypoints and environment-driven config for easier local
runs.
- **Refactors**
- Appended .js to all relative imports and barrel exports across core,
v3 agent, DOM runtimes/locator scripts, caches, handlers, external
clients, and index files.
- Switched examples to ../lib/v3/index.js, fixed nested re-export paths,
and added env-based config (BROWSERBASE_API_KEY, BROWSERBASE_PROJECT_ID,
MODEL_API_KEY) passed into Stagehand/model.
- Updated dotenv to ^17.3.1 in dev deps. No public API changes.
- **Migration**
- Most users importing from package entry points: no changes.
- If importing internal files locally (e.g., examples or custom
integrations), include .js in paths and prefer index.js for directories.
<sup>Written for commit d1d0547daafca47a06a8aa9124d7081c68042270.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1700">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
2026-02-18 12:19:15 -08:00
|
|
|
"dotenv": "^17.3.1",
|
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632)
# why
- unit tests, e2e integration tests, and evals were taking 10~15 minutes
per pr
- finding the specific failing eval or test was tedious and required
reading tons of log output
- we weren't leveraging pnpm or turbo caches or github action's native
chromium install
- tests were flaky because CDP would randomly fail to connect when
github actions runner was overloaded
- e2e bb tests were not actually running against bb, so it was
previously just double-running e2e:local and reporting it as bb
- tsup has been deprecated for a while and should be replaced by esbuild
# what changed
- parallelized all CI tests and broke out evals and tests into
individual matrix jobs
- created unified node+pnpm+turbo setup action that all other actions
call
- verified vanilla chromium is able to launch inside all the e2e local
jobs before running any of our own code to catch resource contention
flakyness
- add proper esm and csj builds with sourcemaps and types for aligned
coverage
- remove tsup dependency
- switch to new evals cli
- add coverage reporting
- fix some flaky tests needed to get everything passing reliably
<img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM"
src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7"
/>
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Speeds up CI and makes tests stable by running Vitest/Playwright/Evals
on the built ESM dist with cached artifacts, verified runner Chromium,
weighted Browserbase routing, and merged coverage. Publishes a unified
ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes
Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test
reporting, addressing Linear STG‑1232.
- **New Features**
- CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium
launch check, region selection, CTRF/V8 uploads); reuse artifacts; use
Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports;
V8 coverage normalization/merge with cancellation safety; push triggers
enabled.
- Tests: run from dist/esm; env reporter; true Browserbase runs with
weighted/randomized regions; hard timeouts for LLM and session
create/CDP connect; faster multi‑click dispatch; higher timeouts and
targeted skips; cleanup helpers; fixes for init‑script,
wait‑for‑selector, and screenshot reliability.
- Build/SDK/SEA: export map (ESM import + CJS require) with
sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain
logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and
exported; getAISDKLanguageModel added; replay metrics now include
method/parameters/result/timestamps/cost.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV
(replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS,
BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS;
optionally set BROWSERBASE_REGION_DISTRIBUTION.
- Remove dotenv; set env vars directly and import
@browserbasehq/stagehand via the export map (avoid dist paths).
<sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types.
CI runs Vitest/Playwright and the new Evals CLI against the built ESM
dist using the runner’s Chromium (launch‑verified), with region‑weighted
Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and
more stable tests (timeouts, Browserbase skips, understudy click/API
fixes).
Speed up CI by running Vitest and Playwright tests in parallel matrices,
add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard
timeouts and better cleanup. Server integration tests now use the
runner’s Chromium for reliability and are folded into the main CI.
- **Refactors**
- Parallelized core unit tests via discover-core-tests/core-unit-tests
(Vitest matrix, fail-fast).
- Parallelized E2E via discover-e2e-tests (Playwright local +
Browserbase) gated on core changes; weighted Browserbase region
selection; verify Chromium/CDP before tests; local uses runner Chromium
via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when
CTRF_JUNIT_PATH is set.
- Unified Lint & Build with Turbo caching and shared artifacts;
tests/evals download and reuse; prepare script skips build in CI; SEA
builds support sourcemaps.
- Enforced configurable timeouts and parallelism via env (LLM_MAX_MS,
BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS,
LOCAL_SESSION_LIMIT_PER_E2E_TEST,
BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and
Browserbase hard timeouts; improved click event dispatch to reduce
remote flakiness.
- Explicitly close Browserbase sessions in tests and evals to free
capacity.
- Added composite actions: setup-node-pnpm-turbo,
verify-chromium-launch, select-browserbase-region, publish-ctrf-report
(JUnit→CTRF and evals), upload-v8-coverage.
- Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter
for CI logs; removed the standalone stagehand-server tests workflow and
ran server tests in the unified CI with Node’s test runner and GitHub
Actions Chromium.
- **New Features**
- Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map;
types from ESM; SEA binaries built from dist with sourcemaps; server SEA
job targets CJS; new build/test/coverage scripts; Node loader maps
package imports to dist/esm; ESM‑safe pathing (fileURLToPath);
ESLint/Prettier tweaks; dotenv removed.
- CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch,
select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses
runner Chromium via CHROME_PATH with --no-sandbox and launch checks;
weighted Browserbase region selection; unique CTRF artifacts; V8
coverage normalized/merged with debug logs and graceful cancellation;
Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled.
- Tests & API: Vitest/Playwright run on dist/esm; new env reporter and
closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and
targeted skips (e.g., console on Browserbase) reduce flakiness; local
launches honor CHROME_PATH; replay metrics now include
parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI
clients moved to lib/v3/external_clients and exported; env‑driven hard
timeouts for LLM ops and Browserbase session create/CDP connect;
understudy API usage fixes.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune
LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and
BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set
BROWSERBASE_REGION_DISTRIBUTION.
- dotenv support is removed; set env vars explicitly in dev and CI.
- Import @browserbasehq/stagehand via the export map; do not reference
dist paths or prior tsup outputs.
<sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
|
|
|
"esbuild": "0.27.2",
|
2026-02-24 14:03:33 -08:00
|
|
|
"eslint": "^10.0.2",
|
2026-01-22 11:49:31 -08:00
|
|
|
"eslint-plugin-security": "^3.0.1",
|
2025-05-03 14:41:15 -07:00
|
|
|
"globals": "^15.13.0",
|
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632)
# why
- unit tests, e2e integration tests, and evals were taking 10~15 minutes
per pr
- finding the specific failing eval or test was tedious and required
reading tons of log output
- we weren't leveraging pnpm or turbo caches or github action's native
chromium install
- tests were flaky because CDP would randomly fail to connect when
github actions runner was overloaded
- e2e bb tests were not actually running against bb, so it was
previously just double-running e2e:local and reporting it as bb
- tsup has been deprecated for a while and should be replaced by esbuild
# what changed
- parallelized all CI tests and broke out evals and tests into
individual matrix jobs
- created unified node+pnpm+turbo setup action that all other actions
call
- verified vanilla chromium is able to launch inside all the e2e local
jobs before running any of our own code to catch resource contention
flakyness
- add proper esm and csj builds with sourcemaps and types for aligned
coverage
- remove tsup dependency
- switch to new evals cli
- add coverage reporting
- fix some flaky tests needed to get everything passing reliably
<img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM"
src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7"
/>
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Speeds up CI and makes tests stable by running Vitest/Playwright/Evals
on the built ESM dist with cached artifacts, verified runner Chromium,
weighted Browserbase routing, and merged coverage. Publishes a unified
ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes
Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test
reporting, addressing Linear STG‑1232.
- **New Features**
- CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium
launch check, region selection, CTRF/V8 uploads); reuse artifacts; use
Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports;
V8 coverage normalization/merge with cancellation safety; push triggers
enabled.
- Tests: run from dist/esm; env reporter; true Browserbase runs with
weighted/randomized regions; hard timeouts for LLM and session
create/CDP connect; faster multi‑click dispatch; higher timeouts and
targeted skips; cleanup helpers; fixes for init‑script,
wait‑for‑selector, and screenshot reliability.
- Build/SDK/SEA: export map (ESM import + CJS require) with
sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain
logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and
exported; getAISDKLanguageModel added; replay metrics now include
method/parameters/result/timestamps/cost.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV
(replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS,
BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS;
optionally set BROWSERBASE_REGION_DISTRIBUTION.
- Remove dotenv; set env vars directly and import
@browserbasehq/stagehand via the export map (avoid dist paths).
<sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types.
CI runs Vitest/Playwright and the new Evals CLI against the built ESM
dist using the runner’s Chromium (launch‑verified), with region‑weighted
Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and
more stable tests (timeouts, Browserbase skips, understudy click/API
fixes).
Speed up CI by running Vitest and Playwright tests in parallel matrices,
add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard
timeouts and better cleanup. Server integration tests now use the
runner’s Chromium for reliability and are folded into the main CI.
- **Refactors**
- Parallelized core unit tests via discover-core-tests/core-unit-tests
(Vitest matrix, fail-fast).
- Parallelized E2E via discover-e2e-tests (Playwright local +
Browserbase) gated on core changes; weighted Browserbase region
selection; verify Chromium/CDP before tests; local uses runner Chromium
via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when
CTRF_JUNIT_PATH is set.
- Unified Lint & Build with Turbo caching and shared artifacts;
tests/evals download and reuse; prepare script skips build in CI; SEA
builds support sourcemaps.
- Enforced configurable timeouts and parallelism via env (LLM_MAX_MS,
BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS,
LOCAL_SESSION_LIMIT_PER_E2E_TEST,
BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and
Browserbase hard timeouts; improved click event dispatch to reduce
remote flakiness.
- Explicitly close Browserbase sessions in tests and evals to free
capacity.
- Added composite actions: setup-node-pnpm-turbo,
verify-chromium-launch, select-browserbase-region, publish-ctrf-report
(JUnit→CTRF and evals), upload-v8-coverage.
- Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter
for CI logs; removed the standalone stagehand-server tests workflow and
ran server tests in the unified CI with Node’s test runner and GitHub
Actions Chromium.
- **New Features**
- Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map;
types from ESM; SEA binaries built from dist with sourcemaps; server SEA
job targets CJS; new build/test/coverage scripts; Node loader maps
package imports to dist/esm; ESM‑safe pathing (fileURLToPath);
ESLint/Prettier tweaks; dotenv removed.
- CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch,
select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses
runner Chromium via CHROME_PATH with --no-sandbox and launch checks;
weighted Browserbase region selection; unique CTRF artifacts; V8
coverage normalized/merged with debug logs and graceful cancellation;
Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled.
- Tests & API: Vitest/Playwright run on dist/esm; new env reporter and
closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and
targeted skips (e.g., console on Browserbase) reduce flakiness; local
launches honor CHROME_PATH; replay metrics now include
parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI
clients moved to lib/v3/external_clients and exported; env‑driven hard
timeouts for LLM ops and Browserbase session create/CDP connect;
understudy API usage fixes.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune
LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and
BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set
BROWSERBASE_REGION_DISTRIBUTION.
- dotenv support is removed; set env vars explicitly in dev and CI.
- Import @browserbasehq/stagehand via the export map; do not reference
dist paths or prior tsup outputs.
<sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
|
|
|
"junit-to-ctrf": "^0.0.14",
|
2025-05-03 14:41:15 -07:00
|
|
|
"prettier": "^3.2.5",
|
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632)
# why
- unit tests, e2e integration tests, and evals were taking 10~15 minutes
per pr
- finding the specific failing eval or test was tedious and required
reading tons of log output
- we weren't leveraging pnpm or turbo caches or github action's native
chromium install
- tests were flaky because CDP would randomly fail to connect when
github actions runner was overloaded
- e2e bb tests were not actually running against bb, so it was
previously just double-running e2e:local and reporting it as bb
- tsup has been deprecated for a while and should be replaced by esbuild
# what changed
- parallelized all CI tests and broke out evals and tests into
individual matrix jobs
- created unified node+pnpm+turbo setup action that all other actions
call
- verified vanilla chromium is able to launch inside all the e2e local
jobs before running any of our own code to catch resource contention
flakyness
- add proper esm and csj builds with sourcemaps and types for aligned
coverage
- remove tsup dependency
- switch to new evals cli
- add coverage reporting
- fix some flaky tests needed to get everything passing reliably
<img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM"
src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7"
/>
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Speeds up CI and makes tests stable by running Vitest/Playwright/Evals
on the built ESM dist with cached artifacts, verified runner Chromium,
weighted Browserbase routing, and merged coverage. Publishes a unified
ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes
Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test
reporting, addressing Linear STG‑1232.
- **New Features**
- CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium
launch check, region selection, CTRF/V8 uploads); reuse artifacts; use
Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports;
V8 coverage normalization/merge with cancellation safety; push triggers
enabled.
- Tests: run from dist/esm; env reporter; true Browserbase runs with
weighted/randomized regions; hard timeouts for LLM and session
create/CDP connect; faster multi‑click dispatch; higher timeouts and
targeted skips; cleanup helpers; fixes for init‑script,
wait‑for‑selector, and screenshot reliability.
- Build/SDK/SEA: export map (ESM import + CJS require) with
sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain
logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and
exported; getAISDKLanguageModel added; replay metrics now include
method/parameters/result/timestamps/cost.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV
(replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS,
BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS;
optionally set BROWSERBASE_REGION_DISTRIBUTION.
- Remove dotenv; set env vars directly and import
@browserbasehq/stagehand via the export map (avoid dist paths).
<sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types.
CI runs Vitest/Playwright and the new Evals CLI against the built ESM
dist using the runner’s Chromium (launch‑verified), with region‑weighted
Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and
more stable tests (timeouts, Browserbase skips, understudy click/API
fixes).
Speed up CI by running Vitest and Playwright tests in parallel matrices,
add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard
timeouts and better cleanup. Server integration tests now use the
runner’s Chromium for reliability and are folded into the main CI.
- **Refactors**
- Parallelized core unit tests via discover-core-tests/core-unit-tests
(Vitest matrix, fail-fast).
- Parallelized E2E via discover-e2e-tests (Playwright local +
Browserbase) gated on core changes; weighted Browserbase region
selection; verify Chromium/CDP before tests; local uses runner Chromium
via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when
CTRF_JUNIT_PATH is set.
- Unified Lint & Build with Turbo caching and shared artifacts;
tests/evals download and reuse; prepare script skips build in CI; SEA
builds support sourcemaps.
- Enforced configurable timeouts and parallelism via env (LLM_MAX_MS,
BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS,
LOCAL_SESSION_LIMIT_PER_E2E_TEST,
BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and
Browserbase hard timeouts; improved click event dispatch to reduce
remote flakiness.
- Explicitly close Browserbase sessions in tests and evals to free
capacity.
- Added composite actions: setup-node-pnpm-turbo,
verify-chromium-launch, select-browserbase-region, publish-ctrf-report
(JUnit→CTRF and evals), upload-v8-coverage.
- Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter
for CI logs; removed the standalone stagehand-server tests workflow and
ran server tests in the unified CI with Node’s test runner and GitHub
Actions Chromium.
- **New Features**
- Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map;
types from ESM; SEA binaries built from dist with sourcemaps; server SEA
job targets CJS; new build/test/coverage scripts; Node loader maps
package imports to dist/esm; ESM‑safe pathing (fileURLToPath);
ESLint/Prettier tweaks; dotenv removed.
- CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch,
select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses
runner Chromium via CHROME_PATH with --no-sandbox and launch checks;
weighted Browserbase region selection; unique CTRF artifacts; V8
coverage normalized/merged with debug logs and graceful cancellation;
Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled.
- Tests & API: Vitest/Playwright run on dist/esm; new env reporter and
closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and
targeted skips (e.g., console on Browserbase) reduce flakiness; local
launches honor CHROME_PATH; replay metrics now include
parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI
clients moved to lib/v3/external_clients and exported; env‑driven hard
timeouts for LLM ops and Browserbase session create/CDP connect;
understudy API usage fixes.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune
LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and
BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set
BROWSERBASE_REGION_DISTRIBUTION.
- dotenv support is removed; set env vars explicitly in dev and CI.
- Import @browserbasehq/stagehand via the export map; do not reference
dist paths or prior tsup outputs.
<sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
|
|
|
"source-map": "^0.7.4",
|
|
|
|
|
"tsx": "^4.19.4",
|
2026-02-19 15:35:37 -08:00
|
|
|
"turbo": "^2.8.10",
|
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632)
# why
- unit tests, e2e integration tests, and evals were taking 10~15 minutes
per pr
- finding the specific failing eval or test was tedious and required
reading tons of log output
- we weren't leveraging pnpm or turbo caches or github action's native
chromium install
- tests were flaky because CDP would randomly fail to connect when
github actions runner was overloaded
- e2e bb tests were not actually running against bb, so it was
previously just double-running e2e:local and reporting it as bb
- tsup has been deprecated for a while and should be replaced by esbuild
# what changed
- parallelized all CI tests and broke out evals and tests into
individual matrix jobs
- created unified node+pnpm+turbo setup action that all other actions
call
- verified vanilla chromium is able to launch inside all the e2e local
jobs before running any of our own code to catch resource contention
flakyness
- add proper esm and csj builds with sourcemaps and types for aligned
coverage
- remove tsup dependency
- switch to new evals cli
- add coverage reporting
- fix some flaky tests needed to get everything passing reliably
<img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM"
src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7"
/>
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Speeds up CI and makes tests stable by running Vitest/Playwright/Evals
on the built ESM dist with cached artifacts, verified runner Chromium,
weighted Browserbase routing, and merged coverage. Publishes a unified
ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes
Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test
reporting, addressing Linear STG‑1232.
- **New Features**
- CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium
launch check, region selection, CTRF/V8 uploads); reuse artifacts; use
Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports;
V8 coverage normalization/merge with cancellation safety; push triggers
enabled.
- Tests: run from dist/esm; env reporter; true Browserbase runs with
weighted/randomized regions; hard timeouts for LLM and session
create/CDP connect; faster multi‑click dispatch; higher timeouts and
targeted skips; cleanup helpers; fixes for init‑script,
wait‑for‑selector, and screenshot reliability.
- Build/SDK/SEA: export map (ESM import + CJS require) with
sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain
logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and
exported; getAISDKLanguageModel added; replay metrics now include
method/parameters/result/timestamps/cost.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV
(replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS,
BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS;
optionally set BROWSERBASE_REGION_DISTRIBUTION.
- Remove dotenv; set env vars directly and import
@browserbasehq/stagehand via the export map (avoid dist paths).
<sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types.
CI runs Vitest/Playwright and the new Evals CLI against the built ESM
dist using the runner’s Chromium (launch‑verified), with region‑weighted
Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and
more stable tests (timeouts, Browserbase skips, understudy click/API
fixes).
Speed up CI by running Vitest and Playwright tests in parallel matrices,
add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard
timeouts and better cleanup. Server integration tests now use the
runner’s Chromium for reliability and are folded into the main CI.
- **Refactors**
- Parallelized core unit tests via discover-core-tests/core-unit-tests
(Vitest matrix, fail-fast).
- Parallelized E2E via discover-e2e-tests (Playwright local +
Browserbase) gated on core changes; weighted Browserbase region
selection; verify Chromium/CDP before tests; local uses runner Chromium
via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when
CTRF_JUNIT_PATH is set.
- Unified Lint & Build with Turbo caching and shared artifacts;
tests/evals download and reuse; prepare script skips build in CI; SEA
builds support sourcemaps.
- Enforced configurable timeouts and parallelism via env (LLM_MAX_MS,
BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS,
LOCAL_SESSION_LIMIT_PER_E2E_TEST,
BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and
Browserbase hard timeouts; improved click event dispatch to reduce
remote flakiness.
- Explicitly close Browserbase sessions in tests and evals to free
capacity.
- Added composite actions: setup-node-pnpm-turbo,
verify-chromium-launch, select-browserbase-region, publish-ctrf-report
(JUnit→CTRF and evals), upload-v8-coverage.
- Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter
for CI logs; removed the standalone stagehand-server tests workflow and
ran server tests in the unified CI with Node’s test runner and GitHub
Actions Chromium.
- **New Features**
- Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map;
types from ESM; SEA binaries built from dist with sourcemaps; server SEA
job targets CJS; new build/test/coverage scripts; Node loader maps
package imports to dist/esm; ESM‑safe pathing (fileURLToPath);
ESLint/Prettier tweaks; dotenv removed.
- CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch,
select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses
runner Chromium via CHROME_PATH with --no-sandbox and launch checks;
weighted Browserbase region selection; unique CTRF artifacts; V8
coverage normalized/merged with debug logs and graceful cancellation;
Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled.
- Tests & API: Vitest/Playwright run on dist/esm; new env reporter and
closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and
targeted skips (e.g., console on Browserbase) reduce flakiness; local
launches honor CHROME_PATH; replay metrics now include
parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI
clients moved to lib/v3/external_clients and exported; env‑driven hard
timeouts for LLM ops and Browserbase session create/CDP connect;
understudy API usage fixes.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune
LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and
BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set
BROWSERBASE_REGION_DISTRIBUTION.
- dotenv support is removed; set env vars explicitly in dev and CI.
- Import @browserbasehq/stagehand via the export map; do not reference
dist paths or prior tsup outputs.
<sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
|
|
|
"typescript": "5.8.3",
|
2026-02-24 14:03:33 -08:00
|
|
|
"typescript-eslint": "^8.56.1"
|
2025-05-03 14:41:15 -07:00
|
|
|
},
|
2024-10-29 04:00:34 -07:00
|
|
|
"repository": {
|
|
|
|
|
"type": "git",
|
|
|
|
|
"url": "git+https://github.com/browserbase/stagehand.git"
|
|
|
|
|
},
|
|
|
|
|
"bugs": {
|
|
|
|
|
"url": "https://github.com/browserbase/stagehand/issues"
|
|
|
|
|
},
|
2025-03-29 17:46:50 -07:00
|
|
|
"homepage": "https://stagehand.dev",
|
|
|
|
|
"overrides": {
|
2026-01-20 18:24:35 -05:00
|
|
|
"whatwg-url": "^14.0.0",
|
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632)
# why
- unit tests, e2e integration tests, and evals were taking 10~15 minutes
per pr
- finding the specific failing eval or test was tedious and required
reading tons of log output
- we weren't leveraging pnpm or turbo caches or github action's native
chromium install
- tests were flaky because CDP would randomly fail to connect when
github actions runner was overloaded
- e2e bb tests were not actually running against bb, so it was
previously just double-running e2e:local and reporting it as bb
- tsup has been deprecated for a while and should be replaced by esbuild
# what changed
- parallelized all CI tests and broke out evals and tests into
individual matrix jobs
- created unified node+pnpm+turbo setup action that all other actions
call
- verified vanilla chromium is able to launch inside all the e2e local
jobs before running any of our own code to catch resource contention
flakyness
- add proper esm and csj builds with sourcemaps and types for aligned
coverage
- remove tsup dependency
- switch to new evals cli
- add coverage reporting
- fix some flaky tests needed to get everything passing reliably
<img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM"
src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7"
/>
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Speeds up CI and makes tests stable by running Vitest/Playwright/Evals
on the built ESM dist with cached artifacts, verified runner Chromium,
weighted Browserbase routing, and merged coverage. Publishes a unified
ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes
Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test
reporting, addressing Linear STG‑1232.
- **New Features**
- CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium
launch check, region selection, CTRF/V8 uploads); reuse artifacts; use
Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports;
V8 coverage normalization/merge with cancellation safety; push triggers
enabled.
- Tests: run from dist/esm; env reporter; true Browserbase runs with
weighted/randomized regions; hard timeouts for LLM and session
create/CDP connect; faster multi‑click dispatch; higher timeouts and
targeted skips; cleanup helpers; fixes for init‑script,
wait‑for‑selector, and screenshot reliability.
- Build/SDK/SEA: export map (ESM import + CJS require) with
sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain
logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and
exported; getAISDKLanguageModel added; replay metrics now include
method/parameters/result/timestamps/cost.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV
(replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS,
BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS;
optionally set BROWSERBASE_REGION_DISTRIBUTION.
- Remove dotenv; set env vars directly and import
@browserbasehq/stagehand via the export map (avoid dist paths).
<sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types.
CI runs Vitest/Playwright and the new Evals CLI against the built ESM
dist using the runner’s Chromium (launch‑verified), with region‑weighted
Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and
more stable tests (timeouts, Browserbase skips, understudy click/API
fixes).
Speed up CI by running Vitest and Playwright tests in parallel matrices,
add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard
timeouts and better cleanup. Server integration tests now use the
runner’s Chromium for reliability and are folded into the main CI.
- **Refactors**
- Parallelized core unit tests via discover-core-tests/core-unit-tests
(Vitest matrix, fail-fast).
- Parallelized E2E via discover-e2e-tests (Playwright local +
Browserbase) gated on core changes; weighted Browserbase region
selection; verify Chromium/CDP before tests; local uses runner Chromium
via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when
CTRF_JUNIT_PATH is set.
- Unified Lint & Build with Turbo caching and shared artifacts;
tests/evals download and reuse; prepare script skips build in CI; SEA
builds support sourcemaps.
- Enforced configurable timeouts and parallelism via env (LLM_MAX_MS,
BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS,
LOCAL_SESSION_LIMIT_PER_E2E_TEST,
BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and
Browserbase hard timeouts; improved click event dispatch to reduce
remote flakiness.
- Explicitly close Browserbase sessions in tests and evals to free
capacity.
- Added composite actions: setup-node-pnpm-turbo,
verify-chromium-launch, select-browserbase-region, publish-ctrf-report
(JUnit→CTRF and evals), upload-v8-coverage.
- Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter
for CI logs; removed the standalone stagehand-server tests workflow and
ran server tests in the unified CI with Node’s test runner and GitHub
Actions Chromium.
- **New Features**
- Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map;
types from ESM; SEA binaries built from dist with sourcemaps; server SEA
job targets CJS; new build/test/coverage scripts; Node loader maps
package imports to dist/esm; ESM‑safe pathing (fileURLToPath);
ESLint/Prettier tweaks; dotenv removed.
- CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch,
select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses
runner Chromium via CHROME_PATH with --no-sandbox and launch checks;
weighted Browserbase region selection; unique CTRF artifacts; V8
coverage normalized/merged with debug logs and graceful cancellation;
Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled.
- Tests & API: Vitest/Playwright run on dist/esm; new env reporter and
closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and
targeted skips (e.g., console on Browserbase) reduce flakiness; local
launches honor CHROME_PATH; replay metrics now include
parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI
clients moved to lib/v3/external_clients and exported; env‑driven hard
timeouts for LLM ops and Browserbase session create/CDP connect;
understudy API usage fixes.
- **Migration**
- Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune
LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and
BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set
BROWSERBASE_REGION_DISTRIBUTION.
- dotenv support is removed; set env vars explicitly in dev and CI.
- Import @browserbasehq/stagehand via the export map; do not reference
dist paths or prior tsup outputs.
<sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
|
|
|
"jwa": "^2.0.1",
|
|
|
|
|
"zod": "4.2.1",
|
|
|
|
|
"tsx": "4.19.4"
|
2025-05-02 09:21:21 -07:00
|
|
|
},
|
2026-02-19 15:35:37 -08:00
|
|
|
"engines": {
|
|
|
|
|
"node": "^20.19.0 || >=22.12.0"
|
|
|
|
|
},
|
2025-05-03 14:41:15 -07:00
|
|
|
"packageManager": "pnpm@9.15.0+sha512.76e2379760a4328ec4415815bcd6628dee727af3779aaa4c914e3944156c4299921a89f976381ee107d41f12cfa4b66681ca9c718f0668fa0831ed4c6d8ba56c"
|
2024-03-20 10:09:26 -07:00
|
|
|
}
|