stagehand/turbo.json

576 lines
13 KiB
JSON
Raw Permalink Normal View History

{
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"$schema": "https://v2-8-10.turborepo.dev/schema.json",
"globalEnv": [
"CI"
],
"globalDependencies": [
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
".github/workflows/ci.yml",
[chore]: add `.js` to the end of all imports (#1700) # why - removes the need to append .js to all imports during esm build process - aligns us with the default standard for esm / esbuild js projects that also need to support cjs - this allows us to seamlessly support ESM, CJS, and TSX with no remapping needed # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Standardized all internal imports and re-exports to include .js extensions to enable native ESM resolution, while keeping CJS and TS working without path remapping. Updated examples to use explicit index.js entrypoints and environment-driven config for easier local runs. - **Refactors** - Appended .js to all relative imports and barrel exports across core, v3 agent, DOM runtimes/locator scripts, caches, handlers, external clients, and index files. - Switched examples to ../lib/v3/index.js, fixed nested re-export paths, and added env-based config (BROWSERBASE_API_KEY, BROWSERBASE_PROJECT_ID, MODEL_API_KEY) passed into Stagehand/model. - Updated dotenv to ^17.3.1 in dev deps. No public API changes. - **Migration** - Most users importing from package entry points: no changes. - If importing internal files locally (e.g., examples or custom integrations), include .js in paths and prefer index.js for directories. <sup>Written for commit d1d0547daafca47a06a8aa9124d7081c68042270. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1700">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-18 12:19:15 -08:00
"package.json",
"packages/*/package.json",
"tsconfig.json",
Fix typechecking for core/tests, fix SEA build for windows, speed up CI (#1721) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Fixes TypeScript across core tests/scripts, finalizes the default export/CLI, and makes the SEA build reliable on Windows. CI splits build and lint, OpenAPI generation no longer runs Prettier, and builds run in parallel across packages for faster pipelines. - **Bug Fixes** - Stricter TS across tests/scripts (tsconfig includes tests + CLI, added @types/jsdom, switched to playwright-core); tightened types (child-frame probes, DocumentReadyState, model utils, source-map Raw types); fixed OpenAPI typing; removed cross-package imports in server tests; added default-export parity test. - Default export and CLI fixed: introduced StagehandDefault mirroring named exports; ESM/CJS indices now export default; corrected package.json CLI paths/types. - SEA build works on Windows: build scripts use pnpm.cmd with better spawn error handling. - List parsing now works from raw source across core/server/evals and maps cleanly to dist outputs. - **Refactors** - CI split into separate Build and Lint jobs; lint cancels the workflow on failure; optional Turbo cache restore; discovery/runner steps no longer depend on the build job. - Parallelized builds: DOM script generators run concurrently; ESM/CJS builds and server/evals build tasks run in parallel; tuned Turbo inputs to reduce noisy rebuilds and improve caching. <sup>Written for commit a25e17c2411b374e8d35c39f6120d242102ff8dc. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1721">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-20 17:09:39 -08:00
"tsconfig.base.json",
"eslint.config.mjs",
".prettierrc",
".prettierignore"
],
"tasks": {
"build": {
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"dependsOn": ["^build"],
"outputs": [
"dist/**"
],
"inputs": [
"**/*.ts",
"**/*.js",
"package.json",
"tsconfig.json",
"tsconfig.*.json",
"!dist/**"
]
},
"@browserbasehq/stagehand#build": {
Fix Windows SEA build (#1734) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Fixes the Windows SEA build and cross‑platform path resolution (including when run via tsx). CI builds, launches, and smoke‑tests Linux, macOS, and Windows binaries (x64/arm64); CJS/ESM outputs are stable. - Bug Fixes - Reusable SEA build workflow with a full OS/arch matrix (incl. Ubuntu 24.04 ARM and Windows 11 ARM), same‑arch launch check against /healthz, selective artifact upload, and an include_sourcemaps toggle (dev/test only; releases exclude sourcemaps). - Centralized runtimePaths helpers across core/server/evals/docs to replace import.meta/fileURLToPath, fix tsx path resolution, normalize absolute paths, and correct supervisorClient CLI resolution and Windows spawning. - Tooling: split DOM script builds into four Turbo tasks for Windows; switch to in‑process esbuild and Node‑invoked turbo/postject; stabilize CJS/ESM via tsc emits plus small CJS wrappers; refine tsconfig includes; add default NODE_ENV/BB_ENV in server env. <sup>Written for commit b0b191f7b0411b6590a79ce3f10b3fba9435b4d7. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1734">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Touches build/CI and packaging code (SEA generation, path resolution) across multiple packages, so failures could break releases or CI on specific platforms despite being largely tooling-focused. > > **Overview** > Fixes cross-platform SEA building by moving `build-server-sea` to a reusable workflow with a multi-OS/arch matrix (including ARM runners), optional artifact/caching inputs, selective artifact upload, and a same-arch smoke test that boots each binary and hits `/healthz`. > > Standardizes path/root resolution via new `runtimePaths` helpers (core/server/evals/docs) and replaces ad-hoc `fileURLToPath(import.meta.url)`/`__dirname` usage across scripts, tests, and `supervisorClient` (also tightening CLI resolution and SEA detection). > > Simplifies build/tooling: core CJS/ESM builds now rely on `tsc` emits plus small CJS wrapper entrypoints (dropping esbuild bundling of core/tests), DOM script generation is split into separate Turbo tasks, server SEA build switches to in-process `esbuild` and Node-invoked `turbo`/`postject`, and server env config now provides default `NODE_ENV`/`BB_ENV` values. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 2e4a07a6d0b54eddb7fea10d079c8062ecba1d6b. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
2026-02-24 10:39:45 -08:00
"dependsOn": [
"^build",
"gen-version",
"build-dom-scripts:dom",
"build-dom-scripts:locator",
"build-dom-scripts:screenshot",
"build-dom-scripts:a11y"
],
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"outputs": [
"dist/**"
],
"inputs": [
"**/*.ts",
"**/*.js",
"package.json",
"tsconfig.json",
"tsconfig.*.json",
"!lib/version.ts",
"!lib/dom/build/**",
"!lib/v3/dom/build/**",
"!dist/**"
]
},
[STG-1536] Split `packages/stagehand/server` into separate server-v3 and server-v4 dirs (#1796) # Follow-up Tasks - [ ] Update stainless SDK custom code for all languages to pull new `stagehand-server-v3-darwin-x64` binary names (`-v3-` added) <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Split the Stagehand API into `packages/server-v3` and `packages/server-v4`, each with its own builds, tests, SEA binaries, and release workflows. Delivers STG-1536 and lets us keep v3 stable while iterating on v4; CI/test discovery and OpenAPI artifacts are versioned. - **Refactors** - Renamed the original server to `packages/server-v3` (`@browserbasehq/stagehand-server-v3`); updated docs and runtime path helpers (now synced across core/docs/evals and both servers), ESLint globs/ignores, scripts/Turbo filters, tests, and Stainless to read `packages/server-v3/openapi.v3.yaml`; v3 SEA binaries use `stagehand-server-v3-*`. - Added `packages/server-v4` (`@browserbasehq/stagehand-server-v4`) with `/v4/**` routes, SSE streaming via `x-stream-response`, LRU/TTL in-memory session store, health/readiness, logging/metrics, `openapi.v4.yaml` + generator, SEA tooling, and v4 integration tests. - CI: path filters, test discovery, and artifacts cover both versions; added `stagehand-server-v4-release.yml` and `stagehand-server-v4-sea-build.yml`; renamed v3 workflows; artifacts include `packages/server-v3/**` and `packages/server-v4/**` dists and OAS. - **Migration** - Replace `packages/server/**` refs with `packages/server-v3/**` or `packages/server-v4/**`. - Use new package filters and binary names: `@browserbasehq/stagehand-server-v3` / `@browserbasehq/stagehand-server-v4`; `stagehand-server-v3-*` / `stagehand-server-v4-*`. - Update OpenAPI consumers to `packages/server-v3/openapi.v3.yaml` or `packages/server-v4/openapi.v4.yaml`. <sup>Written for commit 2b9114ca8097d28ac1567fc67062b24f789a5c2e. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1796">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-03-09 17:32:57 -07:00
"@browserbasehq/stagehand-server-v3#build": {
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"dependsOn": ["^build"],
"outputs": [
"dist/**",
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"openapi.v3.yaml"
],
"inputs": [
"**/*.ts",
"**/*.js",
"package.json",
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"tsconfig.json",
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"tsconfig.*.json",
"!dist/**",
"!openapi.v3.yaml"
]
},
Recover server-v4 drizzle stack onto main (#1917) ## Why The original server-v4 stacked PRs merged into each other after `STG-1614` had already merged into `main`, so the env/test-harness work, Drizzle foundation, and Drizzle schema/controller flow never actually reached `main`. This PR recovers that full stack onto current `main` in one branch. ## What Changed - cherry-picked the three stacked squash-merge commits from `#1896`, `#1897`, and `#1898` onto fresh `main` - preserved the full `server-v4` env/test harness cleanup, Drizzle foundation, DB smoke tests, schema/migration work, and `/v4/llms` controller/service/repo flow - added one small follow-up commit to normalize `pnpm-lock.yaml` against current `main` after applying the recovered stack ## Test Plan - `pnpm install` - `pnpm --filter @browserbasehq/stagehand-server-v4 run lint` - `pnpm --filter @browserbasehq/stagehand-server-v4 run test:unit` - `pnpm --filter @browserbasehq/stagehand-server-v4 run build` - `pnpm --filter @browserbasehq/stagehand-server-v4 run gen:openapi` <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Recovers the server‑v4 Drizzle stack onto `main`, restoring DB‑backed `/v4/llms` and switching the public `LLMId` to UUID. Adds a typed env, clean app bootstrap, and a local `pglite`/Postgres runtime with migrations, tests, and a simpler source‑based test harness. - **New Features** - Drizzle foundation with `pglite` default and Postgres option; new `db:*` scripts via `drizzle-kit`. - Fastify `databasePlugin` and unified DB client; `app.ts` bootstrap with health/readiness and Swagger/OpenAPI; env via `@t3-oss/env-core`. - Tables/relations for LLMs and Stagehand; DB‑derived Zod schemas; first migration; OpenAPI updated for UUID `LLMId` with strict format and pattern. - LLM module (repo/service/controller) backing `/v4/llms`; routes now delegate to the controller; browser session create uses the service and auto‑creates a system default LLM when `llmId` is missing. - Tests run from source via `tsx`; test server script simplified; `turbo` tasks cleaned up; schema cleanup (`z.url()`). - **Migration** - Run: `pnpm --filter @browserbasehq/stagehand-server-v4 db:migrate`. - Local dev uses `pglite` at `~/.stagehand/db/stagehand-v4`. For Postgres set `STAGEHAND_DB_MODE=postgres` and `DATABASE_URL`. - Update clients to treat `LLMId` as a UUID and regenerate from `openapi.v4.yaml` if needed. <sup>Written for commit 1fb861ed27739ad2330a4a0a6daf6e2c4951a575. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1917">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Nick Sweeting <github@sweeting.me> Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-03-30 19:06:13 +02:00
"@browserbasehq/stagehand-server-v3#build:esm-tests": {
"dependsOn": ["^build:esm"],
"outputs": ["dist/tests/**", "dist/src/**"],
"inputs": [
"tests/**/*.ts",
"src/**/*.ts",
"package.json",
"tsconfig.tests.json",
"!dist/tests/**"
]
},
[STG-1536] Split `packages/stagehand/server` into separate server-v3 and server-v4 dirs (#1796) # Follow-up Tasks - [ ] Update stainless SDK custom code for all languages to pull new `stagehand-server-v3-darwin-x64` binary names (`-v3-` added) <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Split the Stagehand API into `packages/server-v3` and `packages/server-v4`, each with its own builds, tests, SEA binaries, and release workflows. Delivers STG-1536 and lets us keep v3 stable while iterating on v4; CI/test discovery and OpenAPI artifacts are versioned. - **Refactors** - Renamed the original server to `packages/server-v3` (`@browserbasehq/stagehand-server-v3`); updated docs and runtime path helpers (now synced across core/docs/evals and both servers), ESLint globs/ignores, scripts/Turbo filters, tests, and Stainless to read `packages/server-v3/openapi.v3.yaml`; v3 SEA binaries use `stagehand-server-v3-*`. - Added `packages/server-v4` (`@browserbasehq/stagehand-server-v4`) with `/v4/**` routes, SSE streaming via `x-stream-response`, LRU/TTL in-memory session store, health/readiness, logging/metrics, `openapi.v4.yaml` + generator, SEA tooling, and v4 integration tests. - CI: path filters, test discovery, and artifacts cover both versions; added `stagehand-server-v4-release.yml` and `stagehand-server-v4-sea-build.yml`; renamed v3 workflows; artifacts include `packages/server-v3/**` and `packages/server-v4/**` dists and OAS. - **Migration** - Replace `packages/server/**` refs with `packages/server-v3/**` or `packages/server-v4/**`. - Use new package filters and binary names: `@browserbasehq/stagehand-server-v3` / `@browserbasehq/stagehand-server-v4`; `stagehand-server-v3-*` / `stagehand-server-v4-*`. - Update OpenAPI consumers to `packages/server-v3/openapi.v3.yaml` or `packages/server-v4/openapi.v4.yaml`. <sup>Written for commit 2b9114ca8097d28ac1567fc67062b24f789a5c2e. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1796">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-03-09 17:32:57 -07:00
"@browserbasehq/stagehand-server-v4#build": {
"dependsOn": ["^build"],
"outputs": [
"dist/**",
"openapi.v4.yaml"
],
"inputs": [
"**/*.ts",
"**/*.js",
"package.json",
"tsconfig.json",
"tsconfig.*.json",
"!dist/**",
"!openapi.v4.yaml"
]
},
"@browserbasehq/browse-cli#build": {
"dependsOn": ["^build"],
"outputs": [
"dist/**"
],
"inputs": [
"**/*.ts",
"**/*.js",
"package.json",
"tsconfig.json",
"tsup.config.ts",
"!dist/**"
]
},
[STG-1536] Split `packages/stagehand/server` into separate server-v3 and server-v4 dirs (#1796) # Follow-up Tasks - [ ] Update stainless SDK custom code for all languages to pull new `stagehand-server-v3-darwin-x64` binary names (`-v3-` added) <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Split the Stagehand API into `packages/server-v3` and `packages/server-v4`, each with its own builds, tests, SEA binaries, and release workflows. Delivers STG-1536 and lets us keep v3 stable while iterating on v4; CI/test discovery and OpenAPI artifacts are versioned. - **Refactors** - Renamed the original server to `packages/server-v3` (`@browserbasehq/stagehand-server-v3`); updated docs and runtime path helpers (now synced across core/docs/evals and both servers), ESLint globs/ignores, scripts/Turbo filters, tests, and Stainless to read `packages/server-v3/openapi.v3.yaml`; v3 SEA binaries use `stagehand-server-v3-*`. - Added `packages/server-v4` (`@browserbasehq/stagehand-server-v4`) with `/v4/**` routes, SSE streaming via `x-stream-response`, LRU/TTL in-memory session store, health/readiness, logging/metrics, `openapi.v4.yaml` + generator, SEA tooling, and v4 integration tests. - CI: path filters, test discovery, and artifacts cover both versions; added `stagehand-server-v4-release.yml` and `stagehand-server-v4-sea-build.yml`; renamed v3 workflows; artifacts include `packages/server-v3/**` and `packages/server-v4/**` dists and OAS. - **Migration** - Replace `packages/server/**` refs with `packages/server-v3/**` or `packages/server-v4/**`. - Use new package filters and binary names: `@browserbasehq/stagehand-server-v3` / `@browserbasehq/stagehand-server-v4`; `stagehand-server-v3-*` / `stagehand-server-v4-*`. - Update OpenAPI consumers to `packages/server-v3/openapi.v3.yaml` or `packages/server-v4/openapi.v4.yaml`. <sup>Written for commit 2b9114ca8097d28ac1567fc67062b24f789a5c2e. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1796">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-03-09 17:32:57 -07:00
"@browserbasehq/stagehand-server-v4#gen:openapi": {
"dependsOn": ["^build:esm"],
"outputs": ["openapi.v4.yaml"],
"inputs": [
"src/**/*.ts",
"scripts/gen-openapi.ts",
"package.json",
"tsconfig.json",
"tsconfig.*.json",
"!openapi.v4.yaml"
]
},
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"@browserbasehq/stagehand#build:esm": {
Fix Windows SEA build (#1734) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Fixes the Windows SEA build and cross‑platform path resolution (including when run via tsx). CI builds, launches, and smoke‑tests Linux, macOS, and Windows binaries (x64/arm64); CJS/ESM outputs are stable. - Bug Fixes - Reusable SEA build workflow with a full OS/arch matrix (incl. Ubuntu 24.04 ARM and Windows 11 ARM), same‑arch launch check against /healthz, selective artifact upload, and an include_sourcemaps toggle (dev/test only; releases exclude sourcemaps). - Centralized runtimePaths helpers across core/server/evals/docs to replace import.meta/fileURLToPath, fix tsx path resolution, normalize absolute paths, and correct supervisorClient CLI resolution and Windows spawning. - Tooling: split DOM script builds into four Turbo tasks for Windows; switch to in‑process esbuild and Node‑invoked turbo/postject; stabilize CJS/ESM via tsc emits plus small CJS wrappers; refine tsconfig includes; add default NODE_ENV/BB_ENV in server env. <sup>Written for commit b0b191f7b0411b6590a79ce3f10b3fba9435b4d7. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1734">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Touches build/CI and packaging code (SEA generation, path resolution) across multiple packages, so failures could break releases or CI on specific platforms despite being largely tooling-focused. > > **Overview** > Fixes cross-platform SEA building by moving `build-server-sea` to a reusable workflow with a multi-OS/arch matrix (including ARM runners), optional artifact/caching inputs, selective artifact upload, and a same-arch smoke test that boots each binary and hits `/healthz`. > > Standardizes path/root resolution via new `runtimePaths` helpers (core/server/evals/docs) and replaces ad-hoc `fileURLToPath(import.meta.url)`/`__dirname` usage across scripts, tests, and `supervisorClient` (also tightening CLI resolution and SEA detection). > > Simplifies build/tooling: core CJS/ESM builds now rely on `tsc` emits plus small CJS wrapper entrypoints (dropping esbuild bundling of core/tests), DOM script generation is split into separate Turbo tasks, server SEA build switches to in-process `esbuild` and Node-invoked `turbo`/`postject`, and server env config now provides default `NODE_ENV`/`BB_ENV` values. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 2e4a07a6d0b54eddb7fea10d079c8062ecba1d6b. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
2026-02-24 10:39:45 -08:00
"dependsOn": [
"^build:esm",
"gen-version",
"build-dom-scripts:dom",
"build-dom-scripts:locator",
"build-dom-scripts:screenshot",
"build-dom-scripts:a11y"
],
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"outputs": [
"dist/esm/**"
],
"inputs": [
"**/*.ts",
"**/*.js",
"package.json",
"tsconfig.json",
"tsconfig.*.json",
"!lib/version.ts",
"!lib/dom/build/**",
"!lib/v3/dom/build/**",
"!dist/**"
]
},
"@browserbasehq/stagehand#build:cjs": {
Fix Windows SEA build (#1734) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Fixes the Windows SEA build and cross‑platform path resolution (including when run via tsx). CI builds, launches, and smoke‑tests Linux, macOS, and Windows binaries (x64/arm64); CJS/ESM outputs are stable. - Bug Fixes - Reusable SEA build workflow with a full OS/arch matrix (incl. Ubuntu 24.04 ARM and Windows 11 ARM), same‑arch launch check against /healthz, selective artifact upload, and an include_sourcemaps toggle (dev/test only; releases exclude sourcemaps). - Centralized runtimePaths helpers across core/server/evals/docs to replace import.meta/fileURLToPath, fix tsx path resolution, normalize absolute paths, and correct supervisorClient CLI resolution and Windows spawning. - Tooling: split DOM script builds into four Turbo tasks for Windows; switch to in‑process esbuild and Node‑invoked turbo/postject; stabilize CJS/ESM via tsc emits plus small CJS wrappers; refine tsconfig includes; add default NODE_ENV/BB_ENV in server env. <sup>Written for commit b0b191f7b0411b6590a79ce3f10b3fba9435b4d7. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1734">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Touches build/CI and packaging code (SEA generation, path resolution) across multiple packages, so failures could break releases or CI on specific platforms despite being largely tooling-focused. > > **Overview** > Fixes cross-platform SEA building by moving `build-server-sea` to a reusable workflow with a multi-OS/arch matrix (including ARM runners), optional artifact/caching inputs, selective artifact upload, and a same-arch smoke test that boots each binary and hits `/healthz`. > > Standardizes path/root resolution via new `runtimePaths` helpers (core/server/evals/docs) and replaces ad-hoc `fileURLToPath(import.meta.url)`/`__dirname` usage across scripts, tests, and `supervisorClient` (also tightening CLI resolution and SEA detection). > > Simplifies build/tooling: core CJS/ESM builds now rely on `tsc` emits plus small CJS wrapper entrypoints (dropping esbuild bundling of core/tests), DOM script generation is split into separate Turbo tasks, server SEA build switches to in-process `esbuild` and Node-invoked `turbo`/`postject`, and server env config now provides default `NODE_ENV`/`BB_ENV` values. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 2e4a07a6d0b54eddb7fea10d079c8062ecba1d6b. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
2026-02-24 10:39:45 -08:00
"dependsOn": [
"^build:cjs",
"gen-version",
"build-dom-scripts:dom",
"build-dom-scripts:locator",
"build-dom-scripts:screenshot",
"build-dom-scripts:a11y"
],
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"outputs": [
"dist/cjs/**"
],
"inputs": [
"**/*.ts",
"**/*.js",
"package.json",
"tsconfig.json",
"tsconfig.*.json",
"!lib/version.ts",
"!lib/dom/build/**",
"!lib/v3/dom/build/**",
"!dist/**"
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
]
},
"gen-version": {
"outputs": ["lib/version.ts"],
"inputs": [
"scripts/gen-version.ts",
"package.json",
"tsconfig.json",
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"tsconfig.*.json",
"!lib/version.ts"
]
},
Fix Windows SEA build (#1734) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Fixes the Windows SEA build and cross‑platform path resolution (including when run via tsx). CI builds, launches, and smoke‑tests Linux, macOS, and Windows binaries (x64/arm64); CJS/ESM outputs are stable. - Bug Fixes - Reusable SEA build workflow with a full OS/arch matrix (incl. Ubuntu 24.04 ARM and Windows 11 ARM), same‑arch launch check against /healthz, selective artifact upload, and an include_sourcemaps toggle (dev/test only; releases exclude sourcemaps). - Centralized runtimePaths helpers across core/server/evals/docs to replace import.meta/fileURLToPath, fix tsx path resolution, normalize absolute paths, and correct supervisorClient CLI resolution and Windows spawning. - Tooling: split DOM script builds into four Turbo tasks for Windows; switch to in‑process esbuild and Node‑invoked turbo/postject; stabilize CJS/ESM via tsc emits plus small CJS wrappers; refine tsconfig includes; add default NODE_ENV/BB_ENV in server env. <sup>Written for commit b0b191f7b0411b6590a79ce3f10b3fba9435b4d7. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1734">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Touches build/CI and packaging code (SEA generation, path resolution) across multiple packages, so failures could break releases or CI on specific platforms despite being largely tooling-focused. > > **Overview** > Fixes cross-platform SEA building by moving `build-server-sea` to a reusable workflow with a multi-OS/arch matrix (including ARM runners), optional artifact/caching inputs, selective artifact upload, and a same-arch smoke test that boots each binary and hits `/healthz`. > > Standardizes path/root resolution via new `runtimePaths` helpers (core/server/evals/docs) and replaces ad-hoc `fileURLToPath(import.meta.url)`/`__dirname` usage across scripts, tests, and `supervisorClient` (also tightening CLI resolution and SEA detection). > > Simplifies build/tooling: core CJS/ESM builds now rely on `tsc` emits plus small CJS wrapper entrypoints (dropping esbuild bundling of core/tests), DOM script generation is split into separate Turbo tasks, server SEA build switches to in-process `esbuild` and Node-invoked `turbo`/`postject`, and server env config now provides default `NODE_ENV`/`BB_ENV` values. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 2e4a07a6d0b54eddb7fea10d079c8062ecba1d6b. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
2026-02-24 10:39:45 -08:00
"build-dom-scripts:dom": {
"outputs": [
Fix Windows SEA build (#1734) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Fixes the Windows SEA build and cross‑platform path resolution (including when run via tsx). CI builds, launches, and smoke‑tests Linux, macOS, and Windows binaries (x64/arm64); CJS/ESM outputs are stable. - Bug Fixes - Reusable SEA build workflow with a full OS/arch matrix (incl. Ubuntu 24.04 ARM and Windows 11 ARM), same‑arch launch check against /healthz, selective artifact upload, and an include_sourcemaps toggle (dev/test only; releases exclude sourcemaps). - Centralized runtimePaths helpers across core/server/evals/docs to replace import.meta/fileURLToPath, fix tsx path resolution, normalize absolute paths, and correct supervisorClient CLI resolution and Windows spawning. - Tooling: split DOM script builds into four Turbo tasks for Windows; switch to in‑process esbuild and Node‑invoked turbo/postject; stabilize CJS/ESM via tsc emits plus small CJS wrappers; refine tsconfig includes; add default NODE_ENV/BB_ENV in server env. <sup>Written for commit b0b191f7b0411b6590a79ce3f10b3fba9435b4d7. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1734">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Touches build/CI and packaging code (SEA generation, path resolution) across multiple packages, so failures could break releases or CI on specific platforms despite being largely tooling-focused. > > **Overview** > Fixes cross-platform SEA building by moving `build-server-sea` to a reusable workflow with a multi-OS/arch matrix (including ARM runners), optional artifact/caching inputs, selective artifact upload, and a same-arch smoke test that boots each binary and hits `/healthz`. > > Standardizes path/root resolution via new `runtimePaths` helpers (core/server/evals/docs) and replaces ad-hoc `fileURLToPath(import.meta.url)`/`__dirname` usage across scripts, tests, and `supervisorClient` (also tightening CLI resolution and SEA detection). > > Simplifies build/tooling: core CJS/ESM builds now rely on `tsc` emits plus small CJS wrapper entrypoints (dropping esbuild bundling of core/tests), DOM script generation is split into separate Turbo tasks, server SEA build switches to in-process `esbuild` and Node-invoked `turbo`/`postject`, and server env config now provides default `NODE_ENV`/`BB_ENV` values. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 2e4a07a6d0b54eddb7fea10d079c8062ecba1d6b. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
2026-02-24 10:39:45 -08:00
"lib/v3/dom/build/v3-index.js",
"lib/v3/dom/build/scriptV3Content.ts",
"lib/v3/dom/build/rerender-index.js",
"lib/v3/dom/build/reRenderScriptContent.ts"
],
"inputs": [
Fix Windows SEA build (#1734) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Fixes the Windows SEA build and cross‑platform path resolution (including when run via tsx). CI builds, launches, and smoke‑tests Linux, macOS, and Windows binaries (x64/arm64); CJS/ESM outputs are stable. - Bug Fixes - Reusable SEA build workflow with a full OS/arch matrix (incl. Ubuntu 24.04 ARM and Windows 11 ARM), same‑arch launch check against /healthz, selective artifact upload, and an include_sourcemaps toggle (dev/test only; releases exclude sourcemaps). - Centralized runtimePaths helpers across core/server/evals/docs to replace import.meta/fileURLToPath, fix tsx path resolution, normalize absolute paths, and correct supervisorClient CLI resolution and Windows spawning. - Tooling: split DOM script builds into four Turbo tasks for Windows; switch to in‑process esbuild and Node‑invoked turbo/postject; stabilize CJS/ESM via tsc emits plus small CJS wrappers; refine tsconfig includes; add default NODE_ENV/BB_ENV in server env. <sup>Written for commit b0b191f7b0411b6590a79ce3f10b3fba9435b4d7. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1734">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Touches build/CI and packaging code (SEA generation, path resolution) across multiple packages, so failures could break releases or CI on specific platforms despite being largely tooling-focused. > > **Overview** > Fixes cross-platform SEA building by moving `build-server-sea` to a reusable workflow with a multi-OS/arch matrix (including ARM runners), optional artifact/caching inputs, selective artifact upload, and a same-arch smoke test that boots each binary and hits `/healthz`. > > Standardizes path/root resolution via new `runtimePaths` helpers (core/server/evals/docs) and replaces ad-hoc `fileURLToPath(import.meta.url)`/`__dirname` usage across scripts, tests, and `supervisorClient` (also tightening CLI resolution and SEA detection). > > Simplifies build/tooling: core CJS/ESM builds now rely on `tsc` emits plus small CJS wrapper entrypoints (dropping esbuild bundling of core/tests), DOM script generation is split into separate Turbo tasks, server SEA build switches to in-process `esbuild` and Node-invoked `turbo`/`postject`, and server env config now provides default `NODE_ENV`/`BB_ENV` values. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 2e4a07a6d0b54eddb7fea10d079c8062ecba1d6b. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
2026-02-24 10:39:45 -08:00
"lib/v3/dom/genDomScripts.ts",
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"lib/v3/dom/**/*.ts",
Fix Windows SEA build (#1734) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Fixes the Windows SEA build and cross‑platform path resolution (including when run via tsx). CI builds, launches, and smoke‑tests Linux, macOS, and Windows binaries (x64/arm64); CJS/ESM outputs are stable. - Bug Fixes - Reusable SEA build workflow with a full OS/arch matrix (incl. Ubuntu 24.04 ARM and Windows 11 ARM), same‑arch launch check against /healthz, selective artifact upload, and an include_sourcemaps toggle (dev/test only; releases exclude sourcemaps). - Centralized runtimePaths helpers across core/server/evals/docs to replace import.meta/fileURLToPath, fix tsx path resolution, normalize absolute paths, and correct supervisorClient CLI resolution and Windows spawning. - Tooling: split DOM script builds into four Turbo tasks for Windows; switch to in‑process esbuild and Node‑invoked turbo/postject; stabilize CJS/ESM via tsc emits plus small CJS wrappers; refine tsconfig includes; add default NODE_ENV/BB_ENV in server env. <sup>Written for commit b0b191f7b0411b6590a79ce3f10b3fba9435b4d7. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1734">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Touches build/CI and packaging code (SEA generation, path resolution) across multiple packages, so failures could break releases or CI on specific platforms despite being largely tooling-focused. > > **Overview** > Fixes cross-platform SEA building by moving `build-server-sea` to a reusable workflow with a multi-OS/arch matrix (including ARM runners), optional artifact/caching inputs, selective artifact upload, and a same-arch smoke test that boots each binary and hits `/healthz`. > > Standardizes path/root resolution via new `runtimePaths` helpers (core/server/evals/docs) and replaces ad-hoc `fileURLToPath(import.meta.url)`/`__dirname` usage across scripts, tests, and `supervisorClient` (also tightening CLI resolution and SEA detection). > > Simplifies build/tooling: core CJS/ESM builds now rely on `tsc` emits plus small CJS wrapper entrypoints (dropping esbuild bundling of core/tests), DOM script generation is split into separate Turbo tasks, server SEA build switches to in-process `esbuild` and Node-invoked `turbo`/`postject`, and server env config now provides default `NODE_ENV`/`BB_ENV` values. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 2e4a07a6d0b54eddb7fea10d079c8062ecba1d6b. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
2026-02-24 10:39:45 -08:00
"!lib/v3/dom/build/**",
"package.json",
"tsconfig.json",
"tsconfig.*.json"
]
},
"build-dom-scripts:locator": {
"outputs": [
"lib/v3/dom/build/locatorScripts.generated.ts"
],
"inputs": [
"lib/v3/dom/genLocatorScripts.ts",
"lib/v3/dom/locatorScripts/**/*.ts",
"!lib/v3/dom/build/**",
"package.json",
"tsconfig.json",
"tsconfig.*.json"
]
},
"build-dom-scripts:screenshot": {
"outputs": [
"lib/v3/dom/build/screenshotScripts.generated.ts"
],
"inputs": [
"lib/v3/dom/genScreenshotScripts.ts",
"lib/v3/dom/screenshotScripts/**/*.ts",
"!lib/v3/dom/build/**",
"package.json",
"tsconfig.json",
"tsconfig.*.json"
]
},
"build-dom-scripts:a11y": {
"outputs": [
"lib/v3/dom/build/a11yScripts.generated.ts"
],
"inputs": [
"lib/v3/dom/genA11yScripts.ts",
"lib/v3/dom/a11yScripts/**/*.ts",
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"!lib/v3/dom/build/**",
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"package.json",
"tsconfig.json",
"tsconfig.*.json"
]
},
"build:esm": {
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"dependsOn": ["^build:esm"],
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"outputs": [
"dist/esm/**"
],
"inputs": [
"**/*.ts",
"**/*.js",
"package.json",
"tsconfig.json",
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"tsconfig.*.json",
"!dist/esm/**"
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
]
},
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"build:server:dist": {
"dependsOn": ["^build:esm"],
"outputs": [
"dist/lib/**",
"dist/routes/**",
"dist/types/**",
"dist/*.js",
"dist/*.js.map",
"dist/*.d.ts"
],
"inputs": [
"src/**/*.ts",
"package.json",
"tsconfig.json",
"tsconfig.*.json",
"!dist/lib/**",
"!dist/routes/**",
"!dist/types/**",
"!dist/*.js",
"!dist/*.js.map",
"!dist/*.d.ts"
]
},
"gen:openapi": {
"dependsOn": ["^build:esm"],
"outputs": ["openapi.v3.yaml"],
"inputs": [
"src/**/*.ts",
"scripts/gen-openapi.ts",
"package.json",
"tsconfig.json",
"tsconfig.*.json",
"!openapi.v3.yaml"
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
]
},
"build:sea:esm": {
"dependsOn": ["^build:esm"],
"outputs": [
"dist/sea/**",
"dist/app.mjs"
],
"env": [
"SEA_BUILD_MODE",
"SEA_TARGET_PLATFORM",
"SEA_TARGET_ARCH",
"SEA_BINARY_NAME"
],
"inputs": [
"src/**/*.ts",
"scripts/**/*.ts",
"package.json",
"tsconfig.json",
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"tsconfig.*.json",
"!dist/sea/**",
"!dist/app.mjs"
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
]
},
"build:sea:cjs": {
"dependsOn": ["^build:cjs"],
"outputs": ["dist/sea/**"],
"env": [
"SEA_BUILD_MODE",
"SEA_TARGET_PLATFORM",
"SEA_TARGET_ARCH",
"SEA_BINARY_NAME"
],
"inputs": [
"src/**/*.ts",
"scripts/**/*.ts",
"package.json",
"tsconfig.json",
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"tsconfig.*.json",
"!dist/sea/**"
]
},
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"build:cjs": {
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"dependsOn": ["^build:cjs"],
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"outputs": [
"dist/cjs/**"
],
"inputs": [
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"**/*.ts",
"**/*.js",
"package.json",
"tsconfig.json",
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"tsconfig.*.json",
"!dist/cjs/**"
]
},
"build:cli": {
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"dependsOn": ["^build:esm"],
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"outputs": ["dist/cli/**"],
"inputs": [
"cli.ts",
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"evals.config.json",
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"scripts/**",
"!dist/cli/**"
]
},
"lint": {
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"dependsOn": ["^build"],
"outputs": [],
"inputs": [
"**/*.ts",
Fix typechecking for core/tests, fix SEA build for windows, speed up CI (#1721) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Fixes TypeScript across core tests/scripts, finalizes the default export/CLI, and makes the SEA build reliable on Windows. CI splits build and lint, OpenAPI generation no longer runs Prettier, and builds run in parallel across packages for faster pipelines. - **Bug Fixes** - Stricter TS across tests/scripts (tsconfig includes tests + CLI, added @types/jsdom, switched to playwright-core); tightened types (child-frame probes, DocumentReadyState, model utils, source-map Raw types); fixed OpenAPI typing; removed cross-package imports in server tests; added default-export parity test. - Default export and CLI fixed: introduced StagehandDefault mirroring named exports; ESM/CJS indices now export default; corrected package.json CLI paths/types. - SEA build works on Windows: build scripts use pnpm.cmd with better spawn error handling. - List parsing now works from raw source across core/server/evals and maps cleanly to dist outputs. - **Refactors** - CI split into separate Build and Lint jobs; lint cancels the workflow on failure; optional Turbo cache restore; discovery/runner steps no longer depend on the build job. - Parallelized builds: DOM script generators run concurrently; ESM/CJS builds and server/evals build tasks run in parallel; tuned Turbo inputs to reduce noisy rebuilds and improve caching. <sup>Written for commit a25e17c2411b374e8d35c39f6120d242102ff8dc. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1721">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-20 17:09:39 -08:00
"**/*.tsx",
"**/*.js",
"**/*.jsx",
Fix typechecking for core/tests, fix SEA build for windows, speed up CI (#1721) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Fixes TypeScript across core tests/scripts, finalizes the default export/CLI, and makes the SEA build reliable on Windows. CI splits build and lint, OpenAPI generation no longer runs Prettier, and builds run in parallel across packages for faster pipelines. - **Bug Fixes** - Stricter TS across tests/scripts (tsconfig includes tests + CLI, added @types/jsdom, switched to playwright-core); tightened types (child-frame probes, DocumentReadyState, model utils, source-map Raw types); fixed OpenAPI typing; removed cross-package imports in server tests; added default-export parity test. - Default export and CLI fixed: introduced StagehandDefault mirroring named exports; ESM/CJS indices now export default; corrected package.json CLI paths/types. - SEA build works on Windows: build scripts use pnpm.cmd with better spawn error handling. - List parsing now works from raw source across core/server/evals and maps cleanly to dist outputs. - **Refactors** - CI split into separate Build and Lint jobs; lint cancels the workflow on failure; optional Turbo cache restore; discovery/runner steps no longer depend on the build job. - Parallelized builds: DOM script generators run concurrently; ESM/CJS builds and server/evals build tasks run in parallel; tuned Turbo inputs to reduce noisy rebuilds and improve caching. <sup>Written for commit a25e17c2411b374e8d35c39f6120d242102ff8dc. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1721">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-20 17:09:39 -08:00
"**/*.mjs",
"**/*.cjs",
"**/*.mts",
"**/*.cts",
"**/*.md",
"**/*.mdx",
"**/*.yml",
"**/*.yaml",
"package.json",
"tsconfig.json",
"tsconfig.*.json",
"!dist/**",
"!node_modules/**",
"!.turbo/**"
]
},
"format": {
"outputs": [],
"cache": false
},
"test": {
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"dependsOn": [],
"outputs": [],
"inputs": [
"**/*.spec.ts",
"**/*.test.ts"
]
},
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"test:core": {
"dependsOn": ["build:esm"],
"outputs": [],
"cache": false,
"env": [
"BROWSERBASE_FLOW_LOGS",
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"VITEST_CONSOLE_REPORTER"
],
"inputs": [
"tests/**/*.ts",
"package.json",
"tsconfig.json",
"tsconfig.*.json"
]
},
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"test:e2e": {
"dependsOn": ["build:esm"],
"outputs": [
"playwright-report/**",
"test-results/**"
],
"cache": false,
"env": [
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"BB_API_KEY",
"BB_PROJECT_ID",
"BROWSERBASE_REGION",
"BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST",
"IFRAME_CHILD_FRAME_TIMEOUT_MS",
"IFRAME_DEBUG",
"IFRAME_POPUP_TIMEOUT_MS",
"IFRAME_POPUP_URL_TIMEOUT_MS",
"KEEP_ALIVE_ACTION_EXIT_TIMEOUT_MS",
"KEEP_ALIVE_BB_INFO_TIMEOUT_MS",
"KEEP_ALIVE_BB_TIMEOUT_MS",
"KEEP_ALIVE_DEBUG",
"KEEP_ALIVE_LOCAL_INFO_TIMEOUT_MS",
"KEEP_ALIVE_LOCAL_TIMEOUT_MS",
"KEEP_ALIVE_STAY_OPEN_MS",
"KEEP_ALIVE_VIEW_MS",
"LOCAL_SESSION_LIMIT_PER_E2E_TEST",
"PLAYWRIGHT_CONSOLE_REPORTER",
"STAGEHAND_BROWSER_TARGET",
"STAGEHAND_API_URL"
],
"passThroughEnv": [
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"ANTHROPIC_API_KEY",
"BROWSERBASE_API_KEY",
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"BROWSERBASE_CDP_CONNECT_MAX_MS",
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"BROWSERBASE_PROJECT_ID",
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"BROWSERBASE_SESSION_CREATE_MAX_MS",
"CHROME_PATH",
"CTRF_JUNIT_PATH",
"GEMINI_API_KEY",
"GOOGLE_GENERATIVE_AI_API_KEY",
"HEADLESS",
"LLM_MAX_MS",
"NODE_OPTIONS",
"NODE_V8_COVERAGE",
"OPENAI_API_KEY",
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"STAGEHAND_SERVER_TARGET"
],
"inputs": [
"lib/v3/tests/**/*.ts",
"package.json",
"tsconfig.json",
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"tsconfig.*.json",
"!playwright-report/**",
"!test-results/**"
]
},
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"test:evals": {
"dependsOn": ["build:esm", "build:cli"],
"outputs": [],
"cache": false,
"env": [
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"AGENT_EVAL_MAX_STEPS",
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"BRAINTRUST_API_KEY",
"EVAL_AGENT_MODELS",
"EVAL_AGENT_MODELS_CUA",
"EVAL_CATEGORIES",
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"EVAL_DATASET",
"EVAL_ENV",
"EVAL_GAIA_FILE",
"EVAL_GAIA_LEVEL",
"EVAL_GAIA_LIMIT",
"EVAL_GAIA_SAMPLE",
"EVAL_MAX_CONCURRENCY",
"EVAL_MAX_K",
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"EVAL_MODELS",
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"EVAL_ONLINEMIND2WEB_LIMIT",
"EVAL_ONLINEMIND2WEB_SAMPLE",
"EVAL_PROVIDER",
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"EVAL_TRIAL_COUNT",
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"EVAL_WEBVOYAGER_LIMIT",
"EVAL_WEBVOYAGER_SAMPLE",
"OP_AUTO_ENV_DISABLE",
"OP_ENV_FILE",
"STAGEHAND_BROWSER_TARGET",
"USE_API"
],
"passThroughEnv": [
"ANTHROPIC_API_KEY",
"BROWSERBASE_API_KEY",
"BROWSERBASE_CDP_CONNECT_MAX_MS",
"BROWSERBASE_PROJECT_ID",
"BROWSERBASE_SESSION_CREATE_MAX_MS",
"CHROME_PATH",
"CTRF_JUNIT_PATH",
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"GEMINI_API_KEY",
"GOOGLE_GENERATIVE_AI_API_KEY",
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"HEADLESS",
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"LLM_MAX_MS",
"NODE_OPTIONS",
"NODE_V8_COVERAGE",
"OPENAI_API_KEY",
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"STAGEHAND_SERVER_TARGET"
],
"inputs": [
"scripts/**",
"cli.ts",
"evals.config.json",
"package.json",
"tsconfig.json",
"tsconfig.*.json"
]
},
"test:server": {
Recover server-v4 drizzle stack onto main (#1917) ## Why The original server-v4 stacked PRs merged into each other after `STG-1614` had already merged into `main`, so the env/test-harness work, Drizzle foundation, and Drizzle schema/controller flow never actually reached `main`. This PR recovers that full stack onto current `main` in one branch. ## What Changed - cherry-picked the three stacked squash-merge commits from `#1896`, `#1897`, and `#1898` onto fresh `main` - preserved the full `server-v4` env/test harness cleanup, Drizzle foundation, DB smoke tests, schema/migration work, and `/v4/llms` controller/service/repo flow - added one small follow-up commit to normalize `pnpm-lock.yaml` against current `main` after applying the recovered stack ## Test Plan - `pnpm install` - `pnpm --filter @browserbasehq/stagehand-server-v4 run lint` - `pnpm --filter @browserbasehq/stagehand-server-v4 run test:unit` - `pnpm --filter @browserbasehq/stagehand-server-v4 run build` - `pnpm --filter @browserbasehq/stagehand-server-v4 run gen:openapi` <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Recovers the server‑v4 Drizzle stack onto `main`, restoring DB‑backed `/v4/llms` and switching the public `LLMId` to UUID. Adds a typed env, clean app bootstrap, and a local `pglite`/Postgres runtime with migrations, tests, and a simpler source‑based test harness. - **New Features** - Drizzle foundation with `pglite` default and Postgres option; new `db:*` scripts via `drizzle-kit`. - Fastify `databasePlugin` and unified DB client; `app.ts` bootstrap with health/readiness and Swagger/OpenAPI; env via `@t3-oss/env-core`. - Tables/relations for LLMs and Stagehand; DB‑derived Zod schemas; first migration; OpenAPI updated for UUID `LLMId` with strict format and pattern. - LLM module (repo/service/controller) backing `/v4/llms`; routes now delegate to the controller; browser session create uses the service and auto‑creates a system default LLM when `llmId` is missing. - Tests run from source via `tsx`; test server script simplified; `turbo` tasks cleaned up; schema cleanup (`z.url()`). - **Migration** - Run: `pnpm --filter @browserbasehq/stagehand-server-v4 db:migrate`. - Local dev uses `pglite` at `~/.stagehand/db/stagehand-v4`. For Postgres set `STAGEHAND_DB_MODE=postgres` and `DATABASE_URL`. - Update clients to treat `LLMId` as a UUID and regenerate from `openapi.v4.yaml` if needed. <sup>Written for commit 1fb861ed27739ad2330a4a0a6daf6e2c4951a575. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1917">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Nick Sweeting <github@sweeting.me> Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-03-30 19:06:13 +02:00
"dependsOn": ["build:server:dist", "build:sea:esm", "build:esm-tests"],
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"outputs": [],
"cache": false,
"env": [
"BB_ENV",
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"NODE_ENV",
"NODE_TEST_CONSOLE_REPORTER",
"NODE_TEST_REPORTER",
"NODE_TEST_REPORTER_DESTINATION",
"SEA_BINARY_NAME",
"STAGEHAND_BASE_URL",
"STAGEHAND_SEA_CACHE_DIR",
"STAGEHAND_TEST_LOCAL_CONNECT_TIMEOUT_MS"
],
"passThroughEnv": [
"ANTHROPIC_API_KEY",
"BROWSERBASE_API_KEY",
"BROWSERBASE_CDP_CONNECT_MAX_MS",
"BROWSERBASE_PROJECT_ID",
"BROWSERBASE_SESSION_CREATE_MAX_MS",
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"CHROME_PATH",
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"CTRF_JUNIT_PATH",
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"GEMINI_API_KEY",
"GOOGLE_GENERATIVE_AI_API_KEY",
Speed up turbo, package.json scripts, and CI configuration (#1712) # why # what changed # test plan <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speed up builds and tests by upgrading Turborepo, tightening cache keys, and refactoring scripts to run from compiled dist (ESM/CJS). CI is faster and more reliable, with ESM-first SEA builds for the server. - **Refactors** - Upgrade Turborepo to 2.8.10 and rewrite turbo.json with explicit inputs/outputs for better caching. - Stronger Turbo cache keys in CI (lockfiles + workspace + package.json + turbo.json + commit), workflow-scoped concurrency, and a 60s session create timeout. - Standardized CI via a composite setup action; build now runs before lint. - Build/test scripts resolve the repo root via fileURLToPath; tests run from compiled dist; add Vitest config for CJS; CJS builds include sourcemaps. - Server: ESM-first SEA build and split server build into dist vs SEA steps with a new build:server:dist script. - Unified test scripts across packages; top-level test runs all suites, and evals tests support source or dist CLI via --cli. - **Migration** - Use Node 20.19+ or 22.12+ (enforced in core). - Run pnpm install to pick up the Turbo upgrade. - Build before tests: pnpm -w build. For SEA tests, also run pnpm -w --filter=@browserbasehq/stagehand-server run build:sea:esm. <sup>Written for commit 843857c13fc4a3975c18b0e44002ae9f55e4dd66. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1712">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-19 15:35:37 -08:00
"HEADLESS",
"LLM_MAX_MS",
[STG-1232] Speed up PR Github Actions checks, add code coverage, fix e2e bb tests running not running in bb mode, remove tsup, generate unified re-used ESM and CJS builds with sourcemaps and types, switch CI to new Evals CLI (#1632) # why - unit tests, e2e integration tests, and evals were taking 10~15 minutes per pr - finding the specific failing eval or test was tedious and required reading tons of log output - we weren't leveraging pnpm or turbo caches or github action's native chromium install - tests were flaky because CDP would randomly fail to connect when github actions runner was overloaded - e2e bb tests were not actually running against bb, so it was previously just double-running e2e:local and reporting it as bb - tsup has been deprecated for a while and should be replaced by esbuild # what changed - parallelized all CI tests and broke out evals and tests into individual matrix jobs - created unified node+pnpm+turbo setup action that all other actions call - verified vanilla chromium is able to launch inside all the e2e local jobs before running any of our own code to catch resource contention flakyness - add proper esm and csj builds with sourcemaps and types for aligned coverage - remove tsup dependency - switch to new evals cli - add coverage reporting - fix some flaky tests needed to get everything passing reliably <img width="1635" height="609" alt="Screenshot 2026-01-26 at 1 04 04 PM" src="https://github.com/user-attachments/assets/24bb2ea9-e182-4cb3-9856-6a77efe921b7" /> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Speeds up CI and makes tests stable by running Vitest/Playwright/Evals on the built ESM dist with cached artifacts, verified runner Chromium, weighted Browserbase routing, and merged coverage. Publishes a unified ESM+CJS SDK with sourcemaps/types and SEA binaries from ESM; fixes Browserbase e2e, switches to the new Evals CLI, and adds flaky‑test reporting, addressing Linear STG‑1232. - **New Features** - CI: split pnpm/Turbo caches; composite actions (setup/caches, Chromium launch check, region selection, CTRF/V8 uploads); reuse artifacts; use Actions Chromium via CHROME_PATH with --no-sandbox; unique CTRF reports; V8 coverage normalization/merge with cancellation safety; push triggers enabled. - Tests: run from dist/esm; env reporter; true Browserbase runs with weighted/randomized regions; hard timeouts for LLM and session create/CDP connect; faster multi‑click dispatch; higher timeouts and targeted skips; cleanup helpers; fixes for init‑script, wait‑for‑selector, and screenshot reliability. - Build/SDK/SEA: export map (ESM import + CJS require) with sourcemaps/types; SEA binaries built from ESM with sourcemaps and plain logs in CI; removed tsup and dotenv; AISdk/CustomOpenAI moved to lib and exported; getAISDKLanguageModel added; replay metrics now include method/parameters/result/timestamps/cost. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase, STAGEHAND_ENV (replaces TEST_ENV), CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - Remove dotenv; set env vars directly and import @browserbasehq/stagehand via the export map (avoid dist paths). <sup>Written for commit 5c76fecf6b9016f4ad5273d4143c3fe747b5d06f. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships an ESM‑first SDK with a CJS entry, unified sourcemaps, and types. CI runs Vitest/Playwright and the new Evals CLI against the built ESM dist using the runner’s Chromium (launch‑verified), with region‑weighted Browserbase routing, normalized V8 coverage, unique CTRF artifacts, and more stable tests (timeouts, Browserbase skips, understudy click/API fixes). Speed up CI by running Vitest and Playwright tests in parallel matrices, add V8/CTRF coverage, and stabilize Browserbase/LLM flows with hard timeouts and better cleanup. Server integration tests now use the runner’s Chromium for reliability and are folded into the main CI. - **Refactors** - Parallelized core unit tests via discover-core-tests/core-unit-tests (Vitest matrix, fail-fast). - Parallelized E2E via discover-e2e-tests (Playwright local + Browserbase) gated on core changes; weighted Browserbase region selection; verify Chromium/CDP before tests; local uses runner Chromium via CHROME_PATH and --no-sandbox; Playwright can emit JUnit when CTRF_JUNIT_PATH is set. - Unified Lint & Build with Turbo caching and shared artifacts; tests/evals download and reuse; prepare script skips build in CI; SEA builds support sourcemaps. - Enforced configurable timeouts and parallelism via env (LLM_MAX_MS, BROWSERBASE_CDP_CONNECT_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, LOCAL_SESSION_LIMIT_PER_E2E_TEST, BROWSERBASE_SESSION_LIMIT_PER_E2E_TEST, EVAL_*); added LLM and Browserbase hard timeouts; improved click event dispatch to reduce remote flakiness. - Explicitly close Browserbase sessions in tests and evals to free capacity. - Added composite actions: setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, publish-ctrf-report (JUnit→CTRF and evals), upload-v8-coverage. - Renamed TEST_ENV to STAGEHAND_ENV across E2E; added an env reporter for CI logs; removed the standalone stagehand-server tests workflow and ran server tests in the unified CI with Node’s test runner and GitHub Actions Chromium. - **New Features** - Build/Exports: ESM in dist/esm and CJS in dist/cjs via export map; types from ESM; SEA binaries built from dist with sourcemaps; server SEA job targets CJS; new build/test/coverage scripts; Node loader maps package imports to dist/esm; ESM‑safe pathing (fileURLToPath); ESLint/Prettier tweaks; dotenv removed. - CI: Composite actions (setup-node-pnpm-turbo, verify-chromium-launch, select-browserbase-region, upload-ctrf-report, upload-v8-coverage); uses runner Chromium via CHROME_PATH with --no-sandbox and launch checks; weighted Browserbase region selection; unique CTRF artifacts; V8 coverage normalized/merged with debug logs and graceful cancellation; Playwright/Vitest discover tests in dist/esm; push/PR triggers enabled. - Tests & API: Vitest/Playwright run on dist/esm; new env reporter and closeV3 helper; tighter multi‑click CDP dispatch; higher timeouts and targeted skips (e.g., console on Browserbase) reduce flakiness; local launches honor CHROME_PATH; replay metrics now include parameters/result/timestamps/cost with parser fixes; AISdk/CustomOpenAI clients moved to lib/v3/external_clients and exported; env‑driven hard timeouts for LLM ops and Browserbase session create/CDP connect; understudy API usage fixes. - **Migration** - Set STAGEHAND_BROWSER_TARGET=local|browserbase and CHROME_PATH; tune LLM_MAX_MS, BROWSERBASE_SESSION_CREATE_MAX_MS, and BROWSERBASE_CDP_CONNECT_MAX_MS; optionally set BROWSERBASE_REGION_DISTRIBUTION. - dotenv support is removed; set env vars explicitly in dev and CI. - Import @browserbasehq/stagehand via the export map; do not reference dist paths or prior tsup outputs. <sup>Written for commit 8119145426a4e6e5cb614dee5b3f2d8fe41a65e9. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1632">Review in cubic</a></sup> <!-- End of auto-generated description by cubic. -->
2026-02-17 17:45:49 -08:00
"NODE_OPTIONS",
"NODE_V8_COVERAGE",
"OPENAI_API_KEY",
"STAGEHAND_BROWSER_TARGET",
"STAGEHAND_SERVER_TARGET"
],
"inputs": [
"test/**/*.ts",
"scripts/**",
"src/**/*.ts",
"package.json",
"tsconfig.json",
"tsconfig.*.json"
]
},
"test:cli": {
"dependsOn": ["build"],
"outputs": [],
"cache": false,
"inputs": [
"tests/**/*.ts",
"src/**/*.ts",
"package.json",
"tsconfig.json",
"vitest.config.ts"
]
},
"docs": {
"persistent": true,
"cache": false
},
"dev": {
"persistent": true,
"cache": false
}
}
}