# why
Some pages do not properly encapsulate the necessary information
required to extract the content requested by the user, and can benefit
from a hybrid approach with vision.
# what changed
- New `extract({ options: { screenshot: true } })` flag. Captures the
current viewport as PNG and sends it as an `image_url` part alongside
the a11y tree text in the extraction LLM call.
- Default is `false` — existing callers see no behavior change.
- Gated to AI SDK clients; throws `StagehandInvalidArgumentError`
otherwise (validated in both the handler and `inference.extract`, with
the handler check running before the screenshot is taken).
- Screenshot capture is bracketed by `ensureTimeRemaining()` so it
respects the extract timeout.
- Wired end-to-end: public `ExtractOptions` type,
`ExtractOptionsSchema`, OpenAPI `ExtractOptions`,
`StagehandAPIClient.extract` wire payload, `V3.extract` →
`ExtractHandler` → `inference.extract` →
`buildExtract{System,User}Prompt`.
# test plan
- `extract-screenshot.test.ts` (new) — multimodal message shape on happy
path; rejection for non-AI SDK clients.
- `timeout-handlers.test.ts` — handler captures viewport when enabled,
skips capture by default, rejects screenshot + non-AI SDK client without
touching `page.screenshot` or the snapshot pipeline.
- `api-client-serialization.test.ts` (renamed from
`api-client-observe-variables.test.ts`) — wire payload preserves
`options.screenshot`.
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Adds a screenshot option to `extract()` that sends the current viewport
screenshot with the accessibility tree to improve results on
visually-driven pages.
- New Features
- `extract({ options: { screenshot: boolean } })` captures a PNG of the
current viewport and includes it with the accessibility tree; defaults
to false; only supported with `aisdk` clients (throws
`StagehandInvalidArgumentError` otherwise).
- Prompts are consolidated and multimodal when enabled: system and user
prompts mention the screenshot and using it with the accessibility tree;
attaches an `image_url` data URI; logs when a screenshot is used.
- Public API, types, and OpenAPI updated (`ExtractOptions.screenshot`);
the API client forwards the flag; tests cover prompt shape, viewport
capture, default-off behavior, timeout guards, client-type validation,
and serialization.
<sup>Written for commit d0c75002884eb42fcebb132c0d333259e8d9efbf.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/2149?utm_source=github">Review
in cubic</a></sup>
<!-- End of auto-generated description by cubic. -->
Stagehand API
The Stagehand is a powerful service that provides a RESTful interface for browser automation and session management using the Browserbase platform. It enables recording, playback, and manipulation of browser sessions with a focus on reliability and performance.
📋 Prerequisites
To run the Stagehand API locally, ensure you have the following installed:
- Node.js
- pnpm
🛠 Installation
- Clone the repository:
git clone https://github.com/browserbase/stagehand/
cd stagehand/packages/server-v3
- Install dependencies:
pnpm install
- Set up environment variables:
cp .env.example .env
-
Configure your
.envfile with the environment variables required bysrc/lib/env.ts(BB environment, API base URLs, etc.). -
pnpm dev