## Summary A 401 or 402 from a reader, or a 401, 402 or 403 from a search vendor, is about the key or the account, not the page or the query, so every call after it meets the same answer. The web tools kept asking: on 2026-09-20 Jina answered HTTP 402 on every `web_fetch` of a session, the tool returned one identical error envelope per call, and the model went on calling it. `web_fetch`, `web_search` and `image_search` now remember a refusal per tool instance (`_VendorRefusal` in `raven/agent/tools/web.py`) and answer later calls without a request: - the `error` string is the same for the refusing call and every paused call (`<Vendor> refused the key (HTTP 402)`), so `failure_class` reads them as one cause and the loop's stop-repeating nudge fires; `is_hard_tool_failure` reads the envelope as deterministic (none of the transient markers appear in the text); - the `detail` says what the status means (401 rejected key, 402 out of credit, 403 not allowed), that this call was not sent, and what the user has to do: the vendor's `tools.web.providers.<vendor>.apiKey` slot, its env var, its sign-up page, or another vendor under `tools.web.<search|fetch>.provider`. The `Tell the user:` framing is deliberate: a model cannot fix a key or a bill, and the incident's model retried instead of saying so; - the pause is per tool rather than per session because the key is shared by every session of the process, so one session's refusal is every session's; per-session marking would send each session into the same 402 once; - all three tools read their key live, in both loops: `WebFetchTool` takes a callable key source the way `WebSearchTool` and `ImageSearchTool` do (main gave it the same source in #548 while this PR was open; the rebase keeps main's version), the main loop hands each of them one through `_live_vendor_key`, and the sub-agent lane through one `live_key` helper over the same resolver on its module `LiveConfig` (it used to hand `web_search` and `image_search` the key it was constructed with, while their envelope told the user the slot is read on the next call). The vendor is live too since #543: the main loop hands the tools a callable `provider`, and the lane runs on the vendor main reads at spawn. That resolver, `live_vendor_key` in `raven/config/live.py`, is one function the main loop's two live readers now delegate to, and it follows `WebToolsConfig.vendor_key` in full: the slot, then the pre-vendor leaf that is Serper's (`tools.web.search.apiKey`) or Jina's (`tools.web.jinaApiKey`, through a new `web_jina_key` reader with the same last-good memory as Serper's), then a revocation only where the file names the vendor with an empty key, then the boot value where the file says nothing about the vendor. Review measured the earlier shape losing a paid Jina key the moment any other vendor's key was set (the schema-backed slot reader answers an empty key for a vendor the subtree does not name, and that empty was read as a revocation before the leaf or the boot value), and a harness-supplied key the same way; both rows now answer what the canonical resolver answers. A key set at `tools.web.providers.<vendor>.apiKey` therefore reaches the next call and lifts the pause without a restart, which is what the envelope tells the user to do; otherwise one real request goes through after `VENDOR_REFUSAL_PAUSE_S` (10 minutes) and re-arms the pause if refused again. The envelope's advice names what each route needs: the config slot and the vendor selection are read from the config file without a restart (per call in the main loop, at spawn for a sub-agent), the env var needs a restart; - each call resolves its (vendor, key) pair once, at its start, through a `_resolve()` on each tool -- and for `web_fetch` that one read also decides the Jina substitution, so a keyed reader whose key is cleared in the file (the edit the refusal text points at) reads through Jina, keyless, from the next call instead of sending an empty credential that could arm no pause (`test_a_keyed_reader_whose_key_is_cleared_falls_back_to_jina_per_call`, and the same through the main loop's wiring) -- and that pair is what the request carries, what the envelope names and what a refusal is recorded against; the request builders and normalisers take the vendor and the key as parameters and read the live properties no more. Read again on the way, a key the user replaced while a request was in flight was recorded as refused without ever having been sent, and the next call was answered from the pause (in the main loop the tool serves every session of the process, so all of them lost it for ten minutes); and with the vendor live, an envelope could name a backend the request never reached, since `WebFetchTool.provider` decides the Jina substitution per access. The pause is keyed on the pair, so another vendor given the same key string is a new request; - which statuses count differs by tool, because what a status can mean differs by endpoint. A search vendor's API is the endpoint, so 401, 402 and 403 all speak of the account: Serper answers a rejected key with 403 (measured against the live endpoint). A reader's 403 speaks of the URL: Jina answers an anonymous request for a domain it has blocked with 403 for every URL under it (measured: `r.jina.ai/https://httpbin.org/status/200` answers 403 with Jina's own `AbuseAlleviationError` envelope, "Anonymous access to domain httpbin.org blocked ... due to previous abuse"), and Firecrawl answers 403 for a site its policy does not scrape ("This website is no longer supported"). So `SEARCH_REFUSAL_STATUSES` is `{401, 402, 403}` and `FETCH_REFUSAL_STATUSES` is `{401, 402}`; a reader's 403 is reported per URL exactly as before this PR; - a request that carried no key is never paused, whatever the status, because there was no key to refuse. That is the default configuration, Jina without a key (`needs_key=False`), where review found that one blocked domain paused `web_fetch` for every URL for ten minutes and told the user to replace a key that does not exist; - statuses outside a tool's set (500, 429, a reader's 403, ...) are reported exactly as before and pause nothing. Not changed: which vendor is configured. The incident's Jina account is out of credit; whether to top it up or select Tavily/Exa under `tools.web.fetch.provider` is a configuration decision this PR only makes visible. Tavily reports an exhausted plan with its own 432 and 433 rather than 402; those stay outside the set and are reported per call. The research sub-agent's web tools are the fork's own classes (`agents/raven-research/.../research_flow/tools/web.py`, which imports only `raven.contracts.tool` and `raven.security.network`): they already classify these statuses as `quota_err`, carry no `_VendorRefusal`, and are untouched, so the fork has no pause to lift and gets none of this PR's behaviour. An earlier revision of this description said its pause lifts on the plugin's rebuild; there is no such pause. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification At head 40893f7f, eight commits on origin/main at a40e911d (rebased once more for the changelog anchor; main's commits since d50a13c1 touch no file this PR touches). Main merged #543 and #548 while this PR was open (a callable `provider` on the three tools, the Jina substitution decided per call, a callable key source on `WebFetchTool`, live vendors and keys handed by the gateway loop); `git range-diff` against 38222344: the four earlier commits replayed with main's version taken where the two had written the same thing (the `WebFetchTool` key source, the gateway's `web_fetch` registration) and no other hunk changed, the fifth commit adapts the pause to the live vendor, the sixth makes the live key reader follow the canonical order (the Jina leaf, and the boot value for a vendor the file says nothing about), the seventh pins a leaf rotated in the file, and the eighth is `ruff format` on one test file: ``` uv run --frozen --python 3.12 --all-extras pytest -q -n 0 -p no:randomly tests/test_agent_tools_web_providers.py tests/test_agent_loop_web_tools.py tests/test_config_live.py tests/test_security_web_ssrf.py 235 passed (17 pre-existing PytestWarnings: asyncio marks on sync tests in that file; the four files, test_config_live.py included since main's live-vendor tests live there) COLORTERM=truecolor uv run --frozen --python 3.12 --all-extras pytest -q (full suite, in its own worktree at this head) 24717 passed, 110 skipped in 260.48s (0:04:20) make lint-python lint-imports lint-deps lint-types # all pass PYTHONPATH=. uv run --frozen --python 3.12 --extra dev python scripts/check_source_language.py origin/main...HEAD # exit 0 make check-commits; PR_TITLE=... make check-pr-title; COMMIT_RANGE=origin/main...HEAD make check-large-files # exit 0 git diff --check origin/main...HEAD; git merge-tree --write-tree HEAD origin/main # clean ``` The reviewer's probe, run through the real `WebFetchTool` (Jina, no key; a transport answering 403 for one host and a page for the others), at the previous head and at this one: ``` previous head 067f6232 (web.py swapped in): PAUSED https://blocked.example/a Jina Reader refused the key (HTTP 403) PAUSED https://example.org/b Jina Reader refused the key (HTTP 403) PAUSED https://example.net/c Jina Reader refused the key (HTTP 403) requests sent: 1 this head: error https://blocked.example/a Jina Reader answered HTTP 403 ok https://example.org/b len=4 ok https://example.net/c len=4 requests sent: 3 ``` Revert-to-red: `raven/agent/tools/web.py` reverted to `origin/main`, or to the previous head, with the tests kept fails at collection (`AttributeError: module 'raven.agent.tools.web' has no attribute 'FETCH_REFUSAL_STATUSES'`, exit 2). Mutation, third round, each mutant applied alone with the file restored between runs (the two web test files, 152 tests at this head, `-n 0`; a mutant that changes a status set also changes how many cases the parametrised tests collect, so the rows give the failures): ``` note() records a refusal from a request that carried no key 2 failed (keyless reader, 401 and 402; its 403 case is held by the fetch set alone) FETCH_REFUSAL_STATUSES keeps 403 5 failed (a reader's 403, all five readers) FETCH_REFUSAL_STATUSES drops 402 3 failed (the 402 pause tests, including the loop-level key rotation) SEARCH_REFUSAL_STATUSES drops 403 1 failed (image_search on Serper's 403) web_fetch built on the search set 5 failed control (nothing mutated) 152 passed ``` Fourth round, one key read per call and the sub-agent lane's two tools (the two web test files, 155 tests, `-n 0`): ``` baseline 155 passed MUTANT web_fetch's note() re-reads the key 1 failed (the in-flight case for web_fetch) MUTANT web_search's note() re-reads the key 1 failed (for web_search) MUTANT image_search's note() re-reads the key 1 failed (for image_search) MUTANT web_fetch's request re-reads the key 1 failed (the request carries a key the pause was not recorded against) MUTANT sub-agent lane: web_search on a boot key 1 failed (the lane test) MUTANT sub-agent lane: image_search on a boot key 1 failed (the lane test) MUTANT vendor switch loses its restart qualifier 34 failed (every pause test, both tools) REVERT web.py + raven_loop.py to reviewed head 428cbe6b 38 failed (the three in-flight cases, the lane test, the 34 wording cases) CONTROL restored 155 passed ``` Fifth round, the live vendor (the two web test files plus `test_config_live.py`, 219 tests, `-n 0`): ``` baseline 219 passed MUTANT the pause ignores the vendor 1 failed MUTANT web_fetch resolves the pair twice (key read again for the request) 1 failed MUTANT web_fetch's envelope names the vendor read again 1 failed MUTANT the vendor switch loses its live qualifier 34 failed REVERT web.py to the rebased head 9c788e0b (before the adaptation) 36 failed CONTROL restored 219 passed ``` Sixth round, the live key reader (the two web test files plus `test_config_live.py`, 232 tests, `-n 0`): ``` baseline 232 passed MUTANT the jina leaf is not consulted (serper's alone, as reviewed) 2 failed MUTANT an empty slot is a revocation whether or not the file names the vendor 1 failed MUTANT the leaf reader forgets its last admitted key on a rejected edit 1 failed MUTANT web_fetch runs on the selected vendor even when its key is gone 3 failed REVERT live.py + schema.py to the reviewed head dbc8ffe0 5 failed CONTROL restored 232 passed ``` The reviewer's measurement of the key reader, `boot` from `WebToolsConfig.vendor_key` and `live` from `live_vendor_key` through the real readers, at the reviewed head and at this one: ``` ### reviewed head dbc8ffe0 tools.web.jinaApiKey only boot='sk-jina-paid' live='sk-jina-paid' same key + providers.serper boot='sk-jina-paid' live='' serper's legacy leaf + providers.firecrawl boot='sk-serper-legacy' live='sk-serper-legacy' a cleared slot beside the serper leaf boot='sk-serper-legacy' live='sk-serper-legacy' harness key, providers names other vendors boot='tv-harness' live='' ### this head 40893f7fb tools.web.jinaApiKey only boot='sk-jina-paid' live='sk-jina-paid' same key + providers.serper boot='sk-jina-paid' live='sk-jina-paid' serper's legacy leaf + providers.firecrawl boot='sk-serper-legacy' live='sk-serper-legacy' a cleared slot beside the serper leaf boot='sk-serper-legacy' live='sk-serper-legacy' harness key, providers names other vendors boot='tv-harness' live='tv-harness' ``` The reviewer's in-flight probe (the real `WebFetchTool` on Jina, a transport answering 402 and rotating the key source while the request is in flight), at the fourth round's previous head and at its head (unchanged by the fifth round: the pair is still read once and the old key is still the one sent and paused): ``` ### reviewed head 428cbe6b key-source reads during one execute(): 4 ['KEY-OLD', 'KEY-OLD', 'KEY-OLD', 'KEY-NEW'] Authorization actually sent : ['KEY-OLD'] next call : answered from the pause, 0 request sent, carrying [] (api_key resolves to KEY-NEW) ### this head 38222344 key-source reads during one execute(): 1 ['KEY-OLD'] Authorization actually sent : ['KEY-OLD'] next call : served, 1 request sent, carrying ['KEY-NEW'] (api_key resolves to KEY-NEW) ``` Earlier rounds, kept for the record; each was measured at the head of its round, and the tests they name are unchanged since except for the parametrisations noted below. First round (131 tests in the two files, at f87ec320): ``` refusals never remembered 25 failed a new key does not lift the pause 1 failed the pause never ends 1 failed a 500 pauses too 19 failed paused calls change the error string 23 failed web_search never short-circuits 8 failed image_search never short-circuits 1 failed web_fetch never short-circuits 16 failed control (nothing mutated) 131 passed ``` Second round, the live key (133 tests, at 145344f0): ``` the four production files reverted to the first-round head 3 failed WebFetchTool ignores its key source 4 failed main loop hands the reader a one-time key 1 failed sub-agent lane hands the reader a one-time key 1 failed live_vendor_key never reads the file 6 failed control 133 passed ``` Tests changed across the rounds: `test_a_new_key_lifts_the_pause_at_once` rotates the key through the callable source the loops hand the tool instead of a private field; the "vendor and status only, never the key" rule for search (was 401) and fetch (was 402) now uses a 500, since a refusal status renders differently on purpose; `test_a_reader_refusing_the_key_pauses_the_tool` is parametrised over the reader set (401, 402) and `test_a_search_vendor_refusing_the_key_pauses_the_tool` over the search set (401, 402, 403). New in the third round: `test_a_keyless_reader_is_never_paused_by_a_status[401|402|403]` (the probe above) and `test_a_readers_403_is_about_the_page_not_the_key` for every reader. Fourth round: `test_the_subagent_lanes_reader_reads_its_key_live_too` became `test_the_subagent_lanes_web_tools_read_their_keys_live_too` and covers all three tools; new: `test_a_key_rotated_while_a_request_is_in_flight_is_tried_before_it_is_paused[web_search|image_search|web_fetch]`; the two pause tests assert the remedy wording. Fifth round: new `test_a_refusal_by_one_vendor_pauses_no_other`; the two pause tests assert the remedy's new wording (the config routes are read without a restart, the env var is not); main's own `test_config_live.py` tests for the callable provider pass unchanged. Sixth round: `TestLiveVendorKeyFollowsTheCanonicalOrder` (the reviewer's three rows and three more against `WebToolsConfig.vendor_key` as the oracle), `test_a_vendor_the_file_says_nothing_about_keeps_its_boot_key`, `test_a_revoked_leaf_is_an_answer_not_a_miss`, `test_a_leaf_the_schema_rejects_dispenses_no_new_answer`, `test_a_keyed_reader_whose_key_is_cleared_falls_back_to_jina_per_call` and `test_a_reader_whose_key_is_cleared_in_the_file_reads_through_jina`. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed (CHANGELOG `Unreleased / Fixed`) ## Risk User-visible behaviour changes: - After a 401 or 402 from a reader, or a 401, 402 or 403 from a search vendor, the tool answers for up to ten minutes (or until the key changes) without contacting the vendor, with an envelope/text that names the cause and the fix. A key set at the vendor's config slot is read on the next call and lifts the pause; a user who instead tops up the same account gets one retry at the window's end. - A request that carried no key never pauses, and a reader's 403 never pauses: on the default configuration (Jina without a key) `web_fetch` behaves exactly as before this PR. - All three tools in both loops read their key from the live config on every call (the main loop's `web_search` and `image_search` already did; `web_fetch` and the sub-agent lane's two now do), and resolve their vendor and key once per call, so the pair a request carried is what its refusal is recorded against and a refusal by one vendor pauses no other; a key removed from the file (an empty slot) is a revocation for them too. - The refusal text says the config slot and the vendor selection under `tools.web.<kind>.provider` are read from the config file without a restart, and the env var on restart. - `WebFetchTool.provider` and `api_key` now both come from one `_resolve()`; their values are what main's properties returned, read once per access rather than the key source twice. - A Jina key held at the pre-vendor leaf `tools.web.jinaApiKey` keeps being sent when a `providers` subtree exists beside it, as it did before this PR and as `WebToolsConfig.vendor_key` resolves it; a key the process booted with is kept for a vendor the file's `providers` subtree does not name, and revoked where the subtree names it with an empty key. - The refusing call's own error text changes from `<Vendor> answered HTTP 402` to `<Vendor> refused the key (HTTP 402)` plus a detail, for the statuses in the tool's set. Anything matching the old text for those statuses would need updating; nothing in the repo does. Rollback: revert the squash commit. The pause is process-local state; no config or stored data changes shape. - [x] Security impact considered (the key is never echoed; the envelope names the config path and env var, not the value) - [x] Backward compatibility considered (tool schemas unchanged; non-refusal statuses render exactly as before) - [x] Rollback path is clear for risky changes ## Related Issues N/A --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
720 lines
30 KiB
Python
720 lines
30 KiB
Python
"""Registration rules for the web tools inside ``_register_default_tools``.
|
|
|
|
``web_search`` needs a Serper key. Registering it without one let the model
|
|
reach for a search it could not run, and the tool's error -- naming a config
|
|
file and an env var -- was relayed to whoever was on the other end of the
|
|
channel. It is withheld instead, on the same terms as the media tools right
|
|
below it.
|
|
|
|
``web_fetch`` is the contrast and is asserted alongside: it works with no key,
|
|
so it is registered unconditionally and must stay that way.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import tempfile
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
|
|
from raven.agent.loop import AgentLoop
|
|
from raven.agent.subagent.backends.raven_loop import RavenLoopBackend
|
|
from raven.agent.tools.registry import ToolRegistry
|
|
from raven.agent.tools.web import ImageSearchTool, WebSearchTool
|
|
from raven.contracts.tool import Tool
|
|
from raven.providers.base import LLMProvider, LLMResponse
|
|
from tests._wiring import wire
|
|
|
|
|
|
class _StubProvider(LLMProvider):
|
|
def __init__(self) -> None:
|
|
super().__init__(api_key="test")
|
|
|
|
async def chat(
|
|
self,
|
|
messages,
|
|
tools=None,
|
|
model=None,
|
|
max_tokens=4096,
|
|
temperature=0.7,
|
|
reasoning_effort=None,
|
|
tool_choice=None,
|
|
):
|
|
return LLMResponse(content="stub", finish_reason="stop")
|
|
|
|
def get_default_model(self) -> str:
|
|
return "stub"
|
|
|
|
|
|
@pytest.fixture
|
|
def workspace():
|
|
with tempfile.TemporaryDirectory() as td:
|
|
yield Path(td)
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _no_ambient_serper_key(monkeypatch: pytest.MonkeyPatch) -> None:
|
|
"""The tool falls back to ``SERPER_API_KEY``, so a developer who exports one
|
|
would otherwise see these tests pass for the wrong reason."""
|
|
monkeypatch.delenv("SERPER_API_KEY", raising=False)
|
|
|
|
|
|
def _loop(workspace: Path, **kw) -> AgentLoop:
|
|
return AgentLoop(provider=_StubProvider(), workspace=workspace, model="stub", **wire(**kw))
|
|
|
|
|
|
def test_web_search_is_withheld_without_a_key(workspace) -> None:
|
|
# Registered but withheld: registration stopped being the gate so a key
|
|
# added while the process runs can surface the tool without a restart.
|
|
loop = _loop(workspace)
|
|
|
|
assert not loop.tools.offers_by_name("web_search"), (
|
|
"offering a search that cannot run makes the model relay the tool's setup error to the user"
|
|
)
|
|
assert loop.tools.has("web_search"), "withheld, not unregistered -- the switch must stay reversible"
|
|
assert loop.tools.offers_by_name("web_fetch"), "web_fetch needs no key and must stay unconditional"
|
|
|
|
|
|
def test_a_configured_key_offers_web_search(workspace) -> None:
|
|
loop = _loop(workspace, search_api_key="sk-serper")
|
|
|
|
assert loop.tools.offers_by_name("web_search")
|
|
|
|
|
|
def test_the_env_var_alone_offers_web_search(workspace, monkeypatch: pytest.MonkeyPatch) -> None:
|
|
# The tool resolves its key at call time from the config value *or*
|
|
# SERPER_API_KEY, so a gate that reads only the config would withdraw the
|
|
# tool from a deploy that exports the variable and configures nothing.
|
|
monkeypatch.setenv("SERPER_API_KEY", "sk-from-env")
|
|
|
|
loop = _loop(workspace)
|
|
|
|
assert loop.tools.offers_by_name("web_search")
|
|
|
|
|
|
#: Every layout a key can be added in, and the vendor selected while it is.
|
|
#: The canonical slot is what the settings page and the wizard write; the
|
|
#: pre-vendor leaf is what an unmigrated config still holds. A reader wired for
|
|
#: the leaf alone left the canonical case -- the common one -- withheld until
|
|
#: the next process, which no caller could tell from the tool being unkeyed.
|
|
_LIVE_KEY_LAYOUTS = [
|
|
("pre-vendor leaf", "serper", {"tools": {"web": {"search": {"apiKey": "sk-added-later"}}}}),
|
|
(
|
|
"canonical serper slot",
|
|
"serper",
|
|
{"tools": {"web": {"providers": {"serper": {"apiKey": "sk-added-later"}}}}},
|
|
),
|
|
(
|
|
"canonical slot of another vendor",
|
|
"tavily",
|
|
{
|
|
"tools": {
|
|
"web": {
|
|
"search": {"provider": "tavily"},
|
|
"providers": {"tavily": {"apiKey": "sk-added-later"}},
|
|
}
|
|
}
|
|
},
|
|
),
|
|
]
|
|
|
|
|
|
@pytest.mark.parametrize(("layout", "vendor", "written"), _LIVE_KEY_LAYOUTS, ids=[c[0] for c in _LIVE_KEY_LAYOUTS])
|
|
def test_a_key_added_after_start_surfaces_web_search(
|
|
workspace, tmp_path: Path, monkeypatch, layout: str, vendor: str, written: dict
|
|
) -> None:
|
|
# The reversibility the registration gate could not give: the user edits the
|
|
# config file, nothing re-registers, and the next assembly reads a different
|
|
# answer -- both the veil and the credential the call then uses.
|
|
monkeypatch.delenv("TAVILY_API_KEY", raising=False)
|
|
cfg = tmp_path / "config.json"
|
|
cfg.write_text(json.dumps({}), encoding="utf-8")
|
|
monkeypatch.setattr("raven.home._current_config_path", cfg)
|
|
loop = _loop(workspace, web_search_provider=vendor)
|
|
assert not loop.tools.offers_by_name("web_search")
|
|
|
|
cfg.write_text(json.dumps(written), encoding="utf-8")
|
|
|
|
assert loop.tools.offers_by_name("web_search"), f"{layout}: the added key never reached the tool"
|
|
assert loop.tools.get("web_search").api_key == "sk-added-later"
|
|
|
|
|
|
def test_a_key_cleared_after_start_withdraws_web_search(workspace, tmp_path: Path, monkeypatch) -> None:
|
|
"""The other direction, and the reason an empty slot is an answer rather
|
|
than a miss: a revoked key must not fall through to the boot value the
|
|
process started with."""
|
|
cfg = tmp_path / "config.json"
|
|
cfg.write_text(json.dumps({"tools": {"web": {"providers": {"serper": {"apiKey": "sk-boot"}}}}}), encoding="utf-8")
|
|
monkeypatch.setattr("raven.home._current_config_path", cfg)
|
|
loop = _loop(workspace, web_provider_keys={"serper": "sk-boot"})
|
|
assert loop.tools.offers_by_name("web_search")
|
|
|
|
cfg.write_text(json.dumps({"tools": {"web": {"providers": {"serper": {"apiKey": ""}}}}}), encoding="utf-8")
|
|
|
|
assert not loop.tools.offers_by_name("web_search")
|
|
assert loop.tools.has("web_search"), "withheld, not unregistered"
|
|
|
|
|
|
def test_the_subagent_loop_applies_the_same_rule(workspace, monkeypatch: pytest.MonkeyPatch) -> None:
|
|
# A sub-agent reaching for a search it cannot run reports the failure to its
|
|
# caller, and that text lands in the parent turn.
|
|
#
|
|
# The backend builds its registry inside `run` and keeps no reference, so the
|
|
# names are observed as they are registered. The collector opens before the
|
|
# backend is constructed and drops nothing on the floor: were a registration
|
|
# ever to happen outside a window, it would land in the previous run's list
|
|
# and be caught, rather than vanishing and leaving an assertion that passes
|
|
# over an empty list.
|
|
registered: list[list[str]] = []
|
|
real = ToolRegistry.register
|
|
|
|
def _spy(self, tool): # noqa: ANN001, ANN202
|
|
real(self, tool)
|
|
assert registered, f"{tool.name} was registered outside a collection window"
|
|
registered[-1].append(tool.name)
|
|
|
|
monkeypatch.setattr(ToolRegistry, "register", _spy)
|
|
|
|
async def _names(**kw) -> list[str]:
|
|
registered.append([])
|
|
backend = RavenLoopBackend(provider=_StubProvider(), model="stub", agent_home=workspace / "home", **kw)
|
|
await backend.run("task", task_id="t1", workspace=workspace, executor=None)
|
|
return registered[-1]
|
|
|
|
import asyncio
|
|
|
|
without = asyncio.run(_names())
|
|
with_key = asyncio.run(_names(search_api_key="sk-serper"))
|
|
|
|
# Baselines first: an empty list would satisfy the "not in" assertion below
|
|
# without proving anything about the gate.
|
|
assert "read_file" in without and "web_fetch" in without, without
|
|
assert "read_file" in with_key and "web_fetch" in with_key, with_key
|
|
assert "web_search" not in without
|
|
assert "web_search" in with_key
|
|
|
|
|
|
def test_the_selected_vendors_key_is_what_offers_web_search(workspace, monkeypatch: pytest.MonkeyPatch) -> None:
|
|
"""The gate asks the tool built for the selected vendor. A Serper key does
|
|
not offer a Tavily search, and Tavily's own env var does.
|
|
|
|
Offered rather than registered: the tool is always registered and withheld
|
|
from the advertised array while no key resolves, so the assertions here are
|
|
about the offer. A gate that named ``SERPER_API_KEY`` itself instead of
|
|
asking the tool passed every case but the first.
|
|
"""
|
|
monkeypatch.delenv("TAVILY_API_KEY", raising=False)
|
|
serper_keyed = _loop(workspace, web_search_provider="tavily", search_api_key="sk-serper")
|
|
assert not serper_keyed.tools.offers_by_name("web_search")
|
|
assert serper_keyed.tools.has("web_search"), "withheld, not unregistered"
|
|
|
|
with_key = _loop(workspace, web_search_provider="tavily", web_provider_keys={"tavily": "tv"})
|
|
assert with_key.tools.offers_by_name("web_search")
|
|
assert with_key.tools.get("web_search").provider == "tavily"
|
|
|
|
monkeypatch.setenv("TAVILY_API_KEY", "tv-env")
|
|
assert _loop(workspace, web_search_provider="tavily").tools.offers_by_name("web_search")
|
|
|
|
|
|
def test_a_keyed_reader_without_a_key_registers_jina_instead(workspace, monkeypatch: pytest.MonkeyPatch) -> None:
|
|
"""web_fetch stays unconditional: the reader that cannot run is replaced by
|
|
the one that needs no key, out loud, rather than offered to fail."""
|
|
monkeypatch.delenv("FIRECRAWL_API_KEY", raising=False)
|
|
loop = _loop(workspace, web_fetch_provider="firecrawl")
|
|
assert loop.tools.get("web_fetch").provider == "jina"
|
|
|
|
keyed = _loop(workspace, web_fetch_provider="firecrawl", web_provider_keys={"firecrawl": "fc"})
|
|
assert keyed.tools.get("web_fetch").provider == "firecrawl"
|
|
assert keyed.tools.get("web_fetch").api_key == "fc"
|
|
|
|
|
|
class _RefusingThenServing:
|
|
"""Stands in for ``httpx.AsyncClient``: refuses the boot key, serves the new one."""
|
|
|
|
def __init__(self) -> None:
|
|
self.keys_seen: list[str] = []
|
|
|
|
def __call__(self, *args, **kwargs):
|
|
return self
|
|
|
|
async def __aenter__(self):
|
|
return self
|
|
|
|
async def __aexit__(self, *exc):
|
|
return None
|
|
|
|
async def post(self, url: str, **kwargs):
|
|
import httpx
|
|
|
|
key = kwargs["headers"]["Authorization"].removeprefix("Bearer ")
|
|
self.keys_seen.append(key)
|
|
request = httpx.Request("POST", url)
|
|
if key == "sk-boot":
|
|
return httpx.Response(402, json={}, request=request)
|
|
return httpx.Response(200, json={"results": [{"url": url, "raw_content": "PAGE"}]}, request=request)
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_reader_key_set_after_a_refusal_reaches_the_next_call(workspace, tmp_path: Path, monkeypatch) -> None:
|
|
"""The refusal tells the user to set a working key at the vendor's slot. The
|
|
tool used to hold the key it was built with, so following that instruction
|
|
did nothing until a restart; the slot is now what the next call reads."""
|
|
monkeypatch.delenv("TAVILY_API_KEY", raising=False)
|
|
cfg = tmp_path / "config.json"
|
|
cfg.write_text(json.dumps({"tools": {"web": {"providers": {"tavily": {"apiKey": "sk-boot"}}}}}), encoding="utf-8")
|
|
monkeypatch.setattr("raven.home._current_config_path", cfg)
|
|
monkeypatch.setattr("raven.agent.tools.web.validate_url_target", lambda url: (True, ""))
|
|
client = _RefusingThenServing()
|
|
monkeypatch.setattr("raven.agent.tools.web.httpx.AsyncClient", client)
|
|
loop = _loop(workspace, web_fetch_provider="tavily", web_provider_keys={"tavily": "sk-boot"})
|
|
tool = loop.tools.get("web_fetch")
|
|
assert tool.provider == "tavily"
|
|
|
|
refused = json.loads(await tool.execute("https://a.example"))
|
|
paused = json.loads(await tool.execute("https://b.example"))
|
|
assert refused["error"] == "Tavily refused the key (HTTP 402)" and paused["paused"] is True
|
|
assert client.keys_seen == ["sk-boot"], "the paused call was not sent"
|
|
|
|
cfg.write_text(
|
|
json.dumps({"tools": {"web": {"providers": {"tavily": {"apiKey": "sk-rotated"}}}}}), encoding="utf-8"
|
|
)
|
|
|
|
served = json.loads(await tool.execute("https://c.example"))
|
|
assert served["text"] == "PAGE"
|
|
assert client.keys_seen == ["sk-boot", "sk-rotated"]
|
|
assert tool.api_key == "sk-rotated"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_reader_whose_key_is_cleared_in_the_file_reads_through_jina(
|
|
workspace, tmp_path: Path, monkeypatch
|
|
) -> None:
|
|
"""Clearing ``tools.web.providers.<vendor>.apiKey`` is the edit the refusal
|
|
text points the user at. The reader registered on that vendor then runs on
|
|
Jina, keyless, from the next call, rather than sending an empty credential."""
|
|
monkeypatch.delenv("TAVILY_API_KEY", raising=False)
|
|
monkeypatch.delenv("JINA_API_KEY", raising=False)
|
|
cfg = tmp_path / "config.json"
|
|
cfg.write_text(json.dumps({"tools": {"web": {"providers": {"tavily": {"apiKey": "sk-boot"}}}}}), encoding="utf-8")
|
|
monkeypatch.setattr("raven.home._current_config_path", cfg)
|
|
monkeypatch.setattr("raven.agent.tools.web.validate_url_target", lambda url: (True, ""))
|
|
sent: list[tuple[str, dict]] = []
|
|
|
|
class _Reader:
|
|
def __call__(self, *a, **k):
|
|
return self
|
|
|
|
async def __aenter__(self):
|
|
return self
|
|
|
|
async def __aexit__(self, *exc):
|
|
return None
|
|
|
|
async def post(self, url, **kwargs):
|
|
import httpx
|
|
|
|
sent.append((url, kwargs.get("headers") or {}))
|
|
payload = {"results": [{"url": url, "raw_content": "TAVILY"}]}
|
|
return httpx.Response(200, json=payload, request=httpx.Request("POST", url))
|
|
|
|
async def get(self, url, **kwargs):
|
|
import httpx
|
|
|
|
sent.append((url, kwargs.get("headers") or {}))
|
|
return httpx.Response(200, text="JINA", request=httpx.Request("GET", url))
|
|
|
|
monkeypatch.setattr("raven.agent.tools.web.httpx.AsyncClient", _Reader())
|
|
loop = _loop(workspace, web_fetch_provider="tavily", web_provider_keys={"tavily": "sk-boot"})
|
|
tool = loop.tools.get("web_fetch")
|
|
|
|
first = json.loads(await tool.execute("https://a.example"))
|
|
cfg.write_text(json.dumps({"tools": {"web": {"providers": {"tavily": {"apiKey": ""}}}}}), encoding="utf-8")
|
|
second = json.loads(await tool.execute("https://b.example"))
|
|
|
|
assert first["extractor"] == "tavily-extract" and sent[0][1]["Authorization"] == "Bearer sk-boot"
|
|
assert second["extractor"] == "jina-reader" and second["text"] == "JINA"
|
|
assert sent[1][0].startswith("https://r.jina.ai/") and "Authorization" not in sent[1][1]
|
|
assert tool.provider == "jina"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_the_subagent_lanes_web_tools_read_their_keys_live_too(tmp_path: Path, monkeypatch) -> None:
|
|
"""All three tools carry the advice to set a key at the config slot, so all
|
|
three read it there; a boot-snapshot key on two of them made that advice
|
|
false for one sub-agent run."""
|
|
from raven.agent.subagent.backends.raven_loop import RavenLoopBackend
|
|
from raven.agent.tools.registry import ToolRegistry
|
|
|
|
monkeypatch.delenv("TAVILY_API_KEY", raising=False)
|
|
cfg = tmp_path / "config.json"
|
|
cfg.write_text(json.dumps({}), encoding="utf-8")
|
|
monkeypatch.setattr("raven.home._current_config_path", cfg)
|
|
registered: list = []
|
|
real = ToolRegistry.register
|
|
|
|
def _spy(self, tool):
|
|
real(self, tool)
|
|
registered.append(tool)
|
|
|
|
monkeypatch.setattr(ToolRegistry, "register", _spy)
|
|
backend = RavenLoopBackend(
|
|
provider=_StubProvider(),
|
|
model="stub",
|
|
agent_home=tmp_path,
|
|
web_search_provider="tavily",
|
|
web_fetch_provider="tavily",
|
|
web_provider_keys={"tavily": "sk-boot"},
|
|
image_search=True,
|
|
)
|
|
await backend.run("task", task_id="t1", workspace=tmp_path, executor=None)
|
|
tools = {t.name: t for t in registered if t.name in ("web_search", "image_search", "web_fetch")}
|
|
assert sorted(tools) == ["image_search", "web_fetch", "web_search"]
|
|
assert {t.provider for t in tools.values()} == {"tavily"}
|
|
assert {t.api_key for t in tools.values()} == {"sk-boot"}
|
|
|
|
cfg.write_text(
|
|
json.dumps({"tools": {"web": {"providers": {"tavily": {"apiKey": "sk-rotated"}}}}}), encoding="utf-8"
|
|
)
|
|
|
|
assert {name: t.api_key for name, t in tools.items()} == {name: "sk-rotated" for name in tools}
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_the_unconfigured_error_names_the_config_actually_in_force(tmp_path: Path) -> None:
|
|
"""Reachable only if the key disappears after registration, but the message
|
|
used to hard-code ``~/.raven/config.json`` and send anyone running with
|
|
``--config`` to edit a file the process never reads."""
|
|
from raven.config.loader import get_config_path, set_config_path
|
|
|
|
before = get_config_path()
|
|
chosen = tmp_path / "elsewhere.json"
|
|
try:
|
|
set_config_path(chosen)
|
|
out = await WebSearchTool().execute(query="anything")
|
|
finally:
|
|
set_config_path(before)
|
|
|
|
assert str(chosen) in out
|
|
assert "~/.raven/config.json" not in out
|
|
|
|
|
|
class _PluginSearch(Tool):
|
|
"""A plugin's own web_search, registered last so it shadows the built-in."""
|
|
|
|
name = "web_search"
|
|
description = "plugin search with its own credential story"
|
|
parameters = {"type": "object", "properties": {}}
|
|
|
|
async def execute(self, **kwargs) -> str:
|
|
return "plugin ran"
|
|
|
|
|
|
def test_a_plugin_shadowing_web_search_is_not_gated_by_the_builtin_config(workspace) -> None:
|
|
# Plugin tools register last precisely so one can replace a built-in by
|
|
# name; the unconfigured gate reads the BUILT-IN's config and must judge
|
|
# only the built-in instance, or it hides the plugin behind a section that
|
|
# says nothing about it.
|
|
plugin = _PluginSearch()
|
|
loop = _loop(workspace, plugin_tools=[plugin])
|
|
|
|
assert loop.tools.get("web_search") is plugin
|
|
assert loop.tools.offers_by_name("web_search"), "the built-in's empty section must not gate the plugin"
|
|
|
|
|
|
def test_a_plugin_subclass_of_web_search_is_not_gated_either(workspace) -> None:
|
|
# The ownership check is identity, not type: an isinstance gate read a
|
|
# plugin SUBCLASS as the built-in itself and withheld it on the built-in's
|
|
# empty section, even though the subclass carries its own credential story.
|
|
class _PluginSubclassSearch(WebSearchTool):
|
|
def __init__(self) -> None:
|
|
super().__init__(api_key="sk-plugin-own")
|
|
|
|
plugin = _PluginSubclassSearch()
|
|
loop = _loop(workspace, plugin_tools=[plugin])
|
|
|
|
assert loop.tools.get("web_search") is plugin
|
|
assert loop.tools.offers_by_name("web_search")
|
|
|
|
|
|
def test_a_subagent_exec_tool_reads_the_live_deny_source(workspace, tmp_path: Path, monkeypatch) -> None:
|
|
"""A tightened permission gates a delegated shell the same call it gates a
|
|
direct one: the sub-agent's gate reads the same live deny source the main
|
|
loop's does, not a construction-time snapshot."""
|
|
import asyncio
|
|
|
|
from raven.agent.tools.shell import ExecTool
|
|
from raven.permissions.turn import start_permission_turn
|
|
|
|
cfg = tmp_path / "config.json"
|
|
cfg.write_text(json.dumps({"tools": {"exec": {"extraDenyPatterns": []}}}), encoding="utf-8")
|
|
monkeypatch.setattr("raven.home._current_config_path", cfg)
|
|
|
|
captured: list[ToolRegistry] = []
|
|
real = ToolRegistry.register
|
|
|
|
def _spy(self, tool, **kw): # noqa: ANN001, ANN202
|
|
real(self, tool, **kw)
|
|
if isinstance(tool, ExecTool):
|
|
captured.append(self)
|
|
|
|
monkeypatch.setattr(ToolRegistry, "register", _spy)
|
|
backend = RavenLoopBackend(provider=_StubProvider(), model="stub", agent_home=workspace / "home")
|
|
asyncio.run(backend.run("task", task_id="t1", workspace=workspace, executor=None))
|
|
assert captured, "the backend registered no ExecTool"
|
|
registry = captured[-1]
|
|
|
|
cfg.write_text(json.dumps({"tools": {"exec": {"extraDenyPatterns": ["\\bosascript\\b"]}}}), encoding="utf-8")
|
|
|
|
start_permission_turn(None, conversation_id="sub", turn_id="t1")
|
|
result = asyncio.run(registry.execute("exec", {"command": "osascript -e beep"}))
|
|
assert "blocked" in str(result)
|
|
|
|
|
|
def test_a_subagent_gate_asks_through_the_parent_turns_responder(workspace, tmp_path: Path, monkeypatch) -> None:
|
|
"""A sub-agent's gate reads the turn's mode instead of pinning full, and
|
|
reaches the responder the parent turn bound (a ContextVar its task
|
|
inherits): a delegated exec in ask mode asks the same human a direct one
|
|
would, rather than running in silence or being refused outright."""
|
|
import asyncio
|
|
|
|
from raven.agent.tools.shell import ExecTool
|
|
from raven.contracts.permissions import ApprovalChoice, ApprovalOutcome
|
|
from raven.permissions.turn import start_permission_turn
|
|
|
|
cfg = tmp_path / "config.json"
|
|
cfg.write_text(json.dumps({"permissions": {"mode": "ask"}}), encoding="utf-8")
|
|
monkeypatch.setattr("raven.home._current_config_path", cfg)
|
|
|
|
captured: list[ToolRegistry] = []
|
|
real = ToolRegistry.register
|
|
|
|
def _spy(self, tool, **kw): # noqa: ANN001, ANN202
|
|
real(self, tool, **kw)
|
|
if isinstance(tool, ExecTool):
|
|
captured.append(self)
|
|
|
|
monkeypatch.setattr(ToolRegistry, "register", _spy)
|
|
backend = RavenLoopBackend(provider=_StubProvider(), model="stub", agent_home=workspace / "home")
|
|
asyncio.run(backend.run("task", task_id="t1", workspace=workspace, executor=None))
|
|
assert captured, "the backend registered no ExecTool"
|
|
registry = captured[-1]
|
|
|
|
class _Click:
|
|
def __init__(self) -> None:
|
|
self.seen: list[str] = []
|
|
|
|
async def await_approval(self, **request): # noqa: ANN003, ANN202
|
|
self.seen.append(request["command"])
|
|
return ApprovalOutcome(choice=ApprovalChoice.DENY)
|
|
|
|
click = _Click()
|
|
start_permission_turn(click, conversation_id="sub", turn_id="t1")
|
|
# A write, not a read: `echo` would run without asking under the read-only
|
|
# default, and this test is about the ask reaching the parent's responder.
|
|
result = asyncio.run(registry.execute("exec", {"command": "touch delegated"}))
|
|
assert click.seen == ["touch delegated"]
|
|
assert "denied" in str(result).lower()
|
|
|
|
|
|
@pytest.mark.parametrize("initially_enabled", [False, True])
|
|
def test_design_image_offer_and_execution_follow_host_selection(tmp_path, initially_enabled):
|
|
from raven.config.live import LiveConfig
|
|
from raven.config.schema import MediaGenConfig
|
|
|
|
host = tmp_path / "host.json"
|
|
selection = {"apiKey": "host-key", "model": "openai/gpt-image-2"}
|
|
|
|
def write(enabled):
|
|
host.write_text(json.dumps({"tools": {"media": {"image": selection if enabled else {}}}}))
|
|
|
|
write(initially_enabled)
|
|
startup = {**(selection if initially_enabled else {}), "selectionConfig": str(host)}
|
|
loop = _loop(tmp_path, media_config=MediaGenConfig.model_validate({"image": startup}))
|
|
loop._live_config = LiveConfig(tmp_path / "absent-render.json")
|
|
tool = loop.tools.get("image_generate")
|
|
for enabled in [initially_enabled, not initially_enabled, initially_enabled]:
|
|
write(enabled)
|
|
assert ("image_generate" not in loop._unconfigured_tool_names()) is enabled
|
|
assert tool._config.api_key == ("host-key" if enabled else "")
|
|
assert tool._config.model == ("openai/gpt-image-2" if enabled else "")
|
|
host.write_text(json.dumps({"tools": {}}))
|
|
assert "image_generate" in loop._unconfigured_tool_names()
|
|
assert not tool._config.api_key
|
|
|
|
|
|
def test_image_search_is_withheld_without_a_serper_key_and_offered_with_one(workspace) -> None:
|
|
"""The Design lane was told to call Serper's image endpoint from `exec` with a key in
|
|
a file nothing named; a live run found no key and generated every picture. A tool,
|
|
gated like web_search, is what the lane -- and the host -- reach for instead."""
|
|
unasked = _loop(workspace, search_api_key="sk-serper")
|
|
assert not unasked.tools.has("image_search"), (
|
|
"a loop that did not ask for pictures has no picture tool, keyed or not"
|
|
)
|
|
|
|
bare = _loop(workspace, image_search=True)
|
|
assert bare.tools.has("image_search") and not bare.tools.offers_by_name("image_search")
|
|
|
|
keyed = _loop(workspace, search_api_key="sk-serper", image_search=True)
|
|
assert keyed.tools.offers_by_name("image_search")
|
|
|
|
|
|
def test_image_search_follows_a_selected_vendor_that_has_an_image_surface(workspace) -> None:
|
|
loop = _loop(workspace, web_search_provider="tavily", web_provider_keys={"tavily": "tv-key"}, image_search=True)
|
|
assert loop.tools.offers_by_name("web_search") and loop.tools.offers_by_name("image_search")
|
|
assert loop.tools.get("image_search").provider == "tavily"
|
|
|
|
|
|
def test_image_search_falls_back_to_serper_when_the_selected_vendor_searches_pages_only(workspace) -> None:
|
|
on_exa = _loop(workspace, web_search_provider="exa", web_provider_keys={"exa": "exa-key"}, image_search=True)
|
|
assert on_exa.tools.offers_by_name("web_search"), "Exa searches pages"
|
|
assert not on_exa.tools.offers_by_name("image_search"), "Exa has no image surface and Serper holds no key"
|
|
assert on_exa.tools.get("image_search").provider == "serper"
|
|
|
|
with_serper = _loop(
|
|
workspace, web_search_provider="exa", web_provider_keys={"exa": "exa-key", "serper": "sk"}, image_search=True
|
|
)
|
|
assert with_serper.tools.offers_by_name("image_search"), "Serper's key opens pictures beside Exa's pages"
|
|
|
|
unkeyed_tavily = _loop(workspace, web_search_provider="tavily", search_api_key="sk-serper", image_search=True)
|
|
assert unkeyed_tavily.tools.get("image_search").provider == "serper", "a selected vendor without a key gives way"
|
|
assert unkeyed_tavily.tools.offers_by_name("image_search")
|
|
|
|
|
|
class _ImageResponse:
|
|
def __init__(self, hits):
|
|
self._hits = hits
|
|
|
|
def raise_for_status(self) -> None:
|
|
return None
|
|
|
|
def json(self):
|
|
return {"images": self._hits}
|
|
|
|
|
|
class _ImageClient:
|
|
asked: list[str] = []
|
|
|
|
def __init__(self, *args, **kwargs) -> None:
|
|
pass
|
|
|
|
async def __aenter__(self):
|
|
return self
|
|
|
|
async def __aexit__(self, *exc) -> None:
|
|
return None
|
|
|
|
async def post(self, url, json=None, headers=None, timeout=None):
|
|
type(self).asked.append(json["q"])
|
|
return _ImageResponse(
|
|
[
|
|
{
|
|
"title": f"{json['q']} wide",
|
|
"imageUrl": f"https://pics.example/{json['q']}.png",
|
|
"imageWidth": 1600,
|
|
"imageHeight": 900,
|
|
"domain": "example.com",
|
|
"link": f"https://example.com/{json['q']}",
|
|
},
|
|
{
|
|
"title": "thumbnail",
|
|
"imageUrl": "https://pics.example/small.png",
|
|
"imageWidth": 200,
|
|
"imageHeight": 150,
|
|
"domain": "example.com",
|
|
},
|
|
]
|
|
)
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_image_search_returns_url_size_and_source_and_drops_what_a_screen_cannot_use(monkeypatch) -> None:
|
|
monkeypatch.setattr("raven.agent.tools.web.httpx.AsyncClient", _ImageClient)
|
|
_ImageClient.asked = []
|
|
tool = ImageSearchTool(api_key="k")
|
|
|
|
said = await tool.execute(queries=["gugong", "tiantan"])
|
|
|
|
assert _ImageClient.asked == ["gugong", "tiantan"]
|
|
assert "Image results for: gugong" in said and "Image results for: tiantan" in said
|
|
assert "https://pics.example/gugong.png" in said and "1600x900px - example.com" in said
|
|
assert "from: https://example.com/gugong" in said
|
|
assert "small.png" not in said, "200px wide is a thumbnail, not a picture"
|
|
assert "Error" in await tool.execute(), "neither form is a search"
|
|
assert "API key not configured" in await ImageSearchTool(api_key=None).execute(query="x")
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
("vendor", "payload"),
|
|
[
|
|
(
|
|
"serpapi",
|
|
{
|
|
"images_results": [
|
|
{
|
|
"title": "gate",
|
|
"original": "https://p.example/g.jpg",
|
|
"original_width": 1600,
|
|
"original_height": 900,
|
|
"source": "example.com",
|
|
"link": "https://example.com/gate",
|
|
}
|
|
]
|
|
},
|
|
),
|
|
(
|
|
"brave",
|
|
{
|
|
"results": [
|
|
{
|
|
"title": "gate",
|
|
"url": "https://example.com/gate",
|
|
"source": "example.com",
|
|
"properties": {"url": "https://p.example/g.jpg", "width": 1600, "height": 900},
|
|
}
|
|
]
|
|
},
|
|
),
|
|
(
|
|
"firecrawl",
|
|
{
|
|
"data": {
|
|
"images": [
|
|
{
|
|
"title": "gate",
|
|
"imageUrl": "https://p.example/g.jpg",
|
|
"imageWidth": 1600,
|
|
"imageHeight": 900,
|
|
"url": "https://example.com/gate",
|
|
"position": 1,
|
|
}
|
|
]
|
|
}
|
|
},
|
|
),
|
|
],
|
|
)
|
|
def test_every_vendor_with_an_image_surface_is_read_into_the_same_hit(vendor, payload) -> None:
|
|
hit = ImageSearchTool(api_key="k", provider=vendor).normalise_hits(payload)[0]
|
|
assert (hit.title, hit.image_url, hit.width, hit.height) == ("gate", "https://p.example/g.jpg", 1600, 900)
|
|
assert hit.source == "example.com" and hit.page == "https://example.com/gate"
|
|
|
|
|
|
def test_tavily_hits_come_without_a_size_and_are_offered_as_such() -> None:
|
|
tool = ImageSearchTool(api_key="k", provider="tavily")
|
|
hits = tool.normalise_hits(
|
|
{
|
|
"images": [
|
|
{"url": "https://cdn.example/a.jpg", "description": "the gate at dusk"},
|
|
"https://cdn.example/b.jpg",
|
|
]
|
|
}
|
|
)
|
|
assert [h.image_url for h in hits] == ["https://cdn.example/a.jpg", "https://cdn.example/b.jpg"]
|
|
assert hits[0].title == "the gate at dusk" and hits[0].width is None and hits[0].source == "cdn.example"
|
|
|
|
|
|
def test_a_vendor_without_an_image_surface_is_refused_at_construction() -> None:
|
|
with pytest.raises(ValueError):
|
|
ImageSearchTool(api_key="k", provider="exa")
|