12 Commits

Author SHA1 Message Date
Shrey Pandya
a110e39265
chore: remove retired Claude 3.5 and 3.7 Sonnet models (#1775)
Fixes CE-731

## Summary
- Remove Claude 3.5 Sonnet (`claude-3-5-sonnet-latest`, `-20241022`, `-20240620`) and Claude 3.7 Sonnet (`claude-3-7-sonnet-latest`, `-20250219`) from all supported model lists
- These models are **retired** by Anthropic — API calls to them will fail
- Replace with `claude-sonnet-4-20250514` across evals, CI, docs, and examples

## What changed (27 files)
- **Core types**: Removed from `model.ts` type union, `agent.ts` CUA models list
- **Provider mappings**: Removed from `LLMProvider.ts`, `AgentProvider.ts`, server `utils.ts`, server `model.ts`
- **Evals/CI**: Updated `taskConfig.ts`, `initV3.ts`, `ci.yml`, `.env.example` to use `claude-sonnet-4-20250514`
- **Tests**: Updated `model-deprecation.test.ts` and `llm-and-agents.test.ts` (513/513 pass)
- **Docs**: Updated all v2 and v3 documentation references (11 `.mdx` files)
- **Other**: Issue template, MCP example

## Context
Per [Anthropic's model deprecations page](https://docs.anthropic.com/en/docs/resources/model-deprecations):
| Model | Retired |
|-------|---------|
| `claude-3-5-sonnet-20240620` | Oct 28, 2025 |
| `claude-3-5-sonnet-20241022` | Oct 28, 2025 |
| `claude-3-7-sonnet-20250219` | Feb 19, 2026 |

## Test plan
- [x] All 513 unit tests pass (`pnpm exec turbo run test:core`)
- [x] `grep` confirms zero remaining references outside CHANGELOG.md (historical)
- [ ] Verify CI passes

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-03-03 10:29:50 -08:00
Miguel
59cca5b39c
Sameel/stg 859 stagehand v3 api support (#1154)
# why

# what changed

# test plan

---------

Co-authored-by: Sameel <sameel.m.arif@gmail.com>
Co-authored-by: Sean McGuire <seanmcguire1@outlook.com>
2025-10-24 21:05:46 -07:00
tkattkat
444da198b5
add agent reasoning to evaluator, improve prompts & paramaterize max … (#1050)
…steps

# why

- Evaluator often is too strict in its evaluation, resulting in false
positives
- max step limits in evals are very brittle and can cause false
negatives on tasks that exceed the current limit
- Evaluator can sometimes view screenshots as a single source of truth
resulting in false negatives. Providing the agents reasoning alongside
these heps mitigate this
# what changed

- paramaterized max steps to allow for easy configuration across all
evals through env
- adjusted evaluator prompting 
- added "agent reasoning" param which can be used to pass in the agents
reasoning alongside bulk screenshots in a format evaluator can
understand well

# test plan
tested locally
2025-09-04 14:25:31 -07:00
Anirudh Kamath
cf7fe665e6
Adding Groq inference support (#536) (#542)
* Adding Groq inference support (#536)

* Adding Groq inference support

* Add Groq model Llama 3.3 70B Versatile

* Add Groq Llama 3.3 70B SpecdDec model and form filling example

- Added new Groq model "groq-llama-3.3-70b-specdec" to supported models
- Created a new example script for form filling with sensible data using Groq
- Updated model type definitions and provider mappings

* Remove Groq form filling example script

* Add Groq model type to LLMClient

* Add Groq client to LLMProvider

* changeset

---------

Co-authored-by: Sankalp Gunturi <sgunturi@protonmail.com>
2025-03-04 19:20:09 -08:00
Anirudh Kamath
44cf7cc9ac
Adding Cerebras inference support (#502) (#523)
* Adding Cerebras inference support (#502)

* Added support for Cerebras models

Full function calling support for both llama-3.3-70b and llama-3.1-8b models
(llama 3.1 8b struggles with tools)

* Added higher performance model as default for example

* Changed Cerebras model prefixes to include provider

* Update CerebrasClient.ts

* Update LLMProvider.ts

* Update LLMClient.ts

* Update model.ts

* Update LLMClient.ts

* Update LLMProvider.ts

* Update LLMProvider.ts

---------

Co-authored-by: Anirudh Kamath <github@kamath.io>

* changeset

---------

Co-authored-by: kwt00 <116334641+kwt00@users.noreply.github.com>
2025-02-25 18:19:03 -08:00
Sameel
33f2b3f8de
stagehand api offloading (#486)
* add api class & types

* simplify stagehand method calls

* revert example

* changeset

* use new syntax in evals

* new extract eval

* allow no observe input

* add api param to constructor

* offload to api

* refresh page after goto and act

* convert zod schema

* use env api url

* add systemprompt to init

* update api responses

* throw error when using api w/ local

* update stream header

* add end call

* fix end & stream

* rm changeset

* revert stagehand config

* Update .env.example

* changeset
2025-02-13 09:38:31 -08:00
Sameel
b902192bc7
customize eval config from env (#283)
* customize eval config from env

* update error messages

* defaults for env

* merges

* experimental eval models setting

* changeset

* add config to CI env
2024-12-12 18:57:39 -05:00
Anirudh Kamath
5a4bca34f6
remove caching (#197) 2024-11-20 20:47:18 -08:00
Filip Michalsky
3588dff576
Fm/remove dom markup (#77)
* add gh actions - WIP

* update dependency package ffor ci

* inject env to gh action

* install playwright deps in ci

* added headless var to env variables

* add headless env injection to evals

* adding more logging to evals

* updated workflows

* use brainstrust gh action

* running gh action

* run all tasks

* uncomment tasks

* turn off costar eval

* add costar back to actions

* error out costar eval

* typo fix

* add gh action for braintrust

* move permissions up in the file

* change logs to JSONL format

* remove undefined function from evals

* fixing braintrust

* revert logs back to strings

* updatge

* update

* trim the DOM

---------

Co-authored-by: Filip Michalsky <filip_michalsky@g.harvard.edu>
2024-09-25 15:14:57 -04:00
Filip Michalsky
65c85f15b1
[WIP] Fm/add anthropic llm (#59)
* add abstraction layer for llm to extend to different LLMs

* remove instructor

* antrhopic wip

* antrhopic is working

---------

Co-authored-by: Filip Michalsky <filip_michalsky@g.harvard.edu>
2024-09-18 20:00:18 -07:00
Paul Klein
1c2103f16e Add evals 2024-05-18 14:14:31 -07:00
Jeremy Press
96d0e89d99 Stagehand prototype 2024-03-24 13:16:24 -07:00