## Summary Raven's host acts as the user's agent manager: it delegates production work, including small edits, to the agent whose ownership or description fits the task -- or, on a turn bound to a Worker Table, to the worker whose brief covers it -- keeps track of the task and its instance, routes follow-ups back to that instance, and gathers the results for the user. Its own tools support briefing, checking and coordination. - Add the resident `multi-agent-coding` skill for choosing coding agents, keeping one instance per task, passing results by file, isolating concurrent edits in worktrees, and handing over work when an agent exhausts its quota. - Keep the new host role authoritative in both the DAG skill's resident description and its full guide. A single task uses `spawn`, or a one-node graph when only the DAG dispatch path is available. - Rename the generic sub-agent to `Raven-General`, with `Raven` and `raven` retained as compatibility aliases. Instance records, handle locks, live activity, cancellation, preferences and RPC lookups use the same canonical identity. Historical spawn and DAG readers normalize names before liveness checks and serialization, so tasks and transcripts join the current instance roster. Concurrent alias calls preserve both turns instead of overwriting one conversation snapshot. The instance registry reads former-name rows under the current name and, on its next write, saves every loaded row under it; transcripts, direct chats and stored graphs stay as written. - Resolve built-in configuration aliases before merging fields: `Raven-General` wins over `Raven`, then `raven`, regardless of file order. Discard older rows wholesale with a warning, preventing stale model, permission or transport settings from leaking into the selected row. - Resume the most recently written message state and instance log across former names. Saved messages take precedence over an empty chat directory; equal timestamps prefer the current name and then the most recent former name. Existing files stay in place. Legacy spawn records, completed DAG manifests and unfinished graphs remain readable through the current name. - Emit the agent roster once at the end of `spawn`'s description. The registry places `run_subagent_dag` and then `spawn` last, including after late MCP or session tools. The graph tool renders the roster itself when `spawn` is unavailable. With a Worker Table bound, dispatch guidance selects worker labels from the live enum; the roster explains underlying capabilities. The tool descriptions and resident DAG guide follow the same rule. - On a turn bound to a Worker Table, such as a Persona session, `spawn` and the DAG nodes offer worker labels alone, and a generated Persona never includes the generic agent. The host role on such a turn routes production work to the worker whose brief covers it, by label, and puts work no worker covers to the user, who can add a worker or give the conversation another Harness, instead of doing it itself or naming an agent outside the enum. Ordinary turns render byte for byte as before. A Harness that `load_playbook` binds mid-turn takes effect after the identity was assembled, so that call's result carries the same routing as this system's trusted note, after the untrusted fence. The coding skill picks worker labels on such a turn, for its first pick and for a quota takeover. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Measured at this head on Linux/Python 3.12.13. - `UV_NO_SYNC=1 uv run --no-sync pytest -q -n 12`: **33,767 passed, 178 skipped, 2 xfailed; 9 failed and 1 error.** Eight of the failures are bound to this box and fail the same way on current main, measured the same day: five provider tests that read the box's proxy environment, a Node.js path case, a LibreOffice conversion, and a fixture that root can read anyway. The ninth, with its teardown error, is the `remote_runs` race tracked in #990; that test passed 5 of 5 runs alone, and its file 58 of 58. - 13 new regression tests for Worker Table routing: 12 fail against the previous head's source and all pass now; the thirteenth checks that a Harness loaded mid-turn leaves the next turn's role plain, and passes on both by design. The end-to-end case drives a real Persona-bound turn: both enums offer `research-a` alone, dispatching `Raven-General` is refused, and the role must name neither that agent nor the roster fallback. - Nine single-construct mutants, each applied to an isolated copy of the tree, each fail at least one new test: the role ignoring the table, the default path not reading the turn's table, the uncovered-work rule removed, the generic fallback kept in the worker routing, the coding skill reverted, and the mid-turn note never added, added on every call, added without a dispatch path, or phrased apart from the role. - The host identity on ordinary turns is byte-identical to the previous head in all 16 rendered shapes: a launched sub-agent process or not, with or without ownership lines, and every combination of the two dispatch tools. - Instance registry probe, run with this head and with main on one synthetic file: reads leave the file byte-identical; one unrelated `upsert_spawn` saves every former-name row as `Raven-General`, keeping the more recently updated of two colliding spellings and the later one in the file on a tie; main then finds none of those rows under `Raven` or `raven`. - `uv run --no-sync python scripts/coverage_gate.py --coverage-json <coverage.json> diff --base-ref origin/main --threshold 90`, over a coverage run of 225 test files (8,952 passed): **99.40%**, 167/168 executable changed lines covered. - `UV_NO_SYNC=1 make lint-types lint-imports check-large-files check-source-language COMMIT_RANGE=origin/main...HEAD`: passed; all 10 import contracts kept. - `UV_NO_SYNC=1 make check-commits`: both commit-message validators passed; each also refused a dangling control commit with a malformed subject. - `ruff check` and `ruff format --check` over the branch's Python files passed. `node ui-tui/scripts/gen-i18n.mjs --check` passed after the rebase onto #1015, which also changed the i18n catalogue. - `uv run --no-sync pre-commit run --from-ref origin/main --to-ref HEAD`: every hook passed except the whole-TUI Prettier hook, which reports only `newInstancePicker.tsx` and `subagentsHub.tsx`; both are byte-identical to current main. - Live-model replay and native Windows/macOS testing were not repeated for these fixes. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed ## Risk - The host delegates work it previously performed itself, including small scripts. Task-to-instance routing remains prompt-based; a task can still stay with the host after it has already answered a follow-up itself. - In a conversation bound to a Persona or another Harness with workers, work no worker's brief covers now waits for the user to add a worker or change the conversation's Harness, rather than being carried out by the host. - Old agent spellings remain accepted. The instance registry's first write after the upgrade, of any row for any session, saves every former-name row in `subagent_instances.json` as `Raven-General`, and keeps only the more recently updated row where two spellings of one instance collide. Builds from before the rename cannot look those rows up, so reverting the squash commit alone does not restore their instance bindings: that needs the registry file as it stood before that first write. Renaming rows back by hand cannot recover a colliding row the write dropped. Transcripts, direct chats and stored graphs are not rewritten. - When multiple generic-agent configuration spellings coexist, only the newest row contributes overrides. Fields supplied exclusively by an older row are intentionally discarded. - Moving the dispatch tools changes the tool-array cache prefix once after upgrade. - A plugin participant that removes `spawn` through `select_tools` can still leave the graph tool pointing at an absent roster, because that selection occurs after the registry's offer check. No bundled participant does this. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues N/A
977 lines
47 KiB
Python
977 lines
47 KiB
Python
"""The five SegmentBuilders, tested in isolation.
|
|
|
|
Each builder is fed a fake :class:`AssemblyContext` and asserted to
|
|
reproduce the segment its old inline block in ``ContextBuilder`` emitted.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import types
|
|
from pathlib import Path
|
|
|
|
from raven.agent.context import ContextBuilder
|
|
from raven.agent.subagent.builtin_agents import GENERIC_AGENT
|
|
from raven.context_engine.segments import (
|
|
ActiveSkillsSegmentBuilder,
|
|
BootstrapSegmentBuilder,
|
|
IdentitySegmentBuilder,
|
|
MemorySegmentBuilder,
|
|
SkillsSegmentBuilder,
|
|
render,
|
|
)
|
|
from raven.contracts.assembled import TokenBudget
|
|
from raven.contracts.context import AssemblyContext
|
|
from raven.contracts.memory import Memory
|
|
from raven.memory_engine.skill_forge import RouterHit, SkillForgeRouter
|
|
|
|
|
|
def _ctx(tmp_path: Path, msg: str = "hi", session=None, **over) -> AssemblyContext:
|
|
return AssemblyContext(
|
|
session_key="s",
|
|
current_message=msg,
|
|
media=None,
|
|
channel=None,
|
|
chat_id=None,
|
|
session_messages=session or [],
|
|
budget=TokenBudget(100_000, 4_000, 2_000, 1_000, 93_000),
|
|
**over,
|
|
)
|
|
|
|
|
|
class _Backend:
|
|
def __init__(self, mems):
|
|
self._mems = mems
|
|
self.calls = []
|
|
|
|
async def recall(self, query, *, user_id=None, agent_id=None, top_k):
|
|
self.calls.append(
|
|
{
|
|
"query": query,
|
|
"user_id": user_id,
|
|
"agent_id": agent_id,
|
|
"top_k": top_k,
|
|
}
|
|
)
|
|
return list(self._mems)
|
|
|
|
|
|
def _tool_defs(*names: str):
|
|
"""A ``get_tool_definitions`` callable in OpenAI function-call shape."""
|
|
return lambda: [{"type": "function", "function": {"name": n, "parameters": {}}} for n in names]
|
|
|
|
|
|
class _Source:
|
|
name = "local"
|
|
weight = 1.0
|
|
|
|
def __init__(self, hits):
|
|
self._hits = hits
|
|
|
|
async def search(self, query, history, k):
|
|
return list(self._hits)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _provider_config(model: str, provider: str, api_key: str | None = None, api_base: str | None = None):
|
|
"""A config stand-in exposing exactly what _resolved_model_id reads."""
|
|
cfg = types.SimpleNamespace(agents=types.SimpleNamespace(defaults=types.SimpleNamespace(model=model)))
|
|
cfg.get_provider_name = lambda m=None: provider
|
|
cfg.get_api_key = lambda m=None: api_key
|
|
cfg.get_api_base = lambda m=None: api_base
|
|
return cfg
|
|
|
|
|
|
class TestIdentityBootstrap:
|
|
async def test_identity_matches_legacy(self, tmp_path: Path) -> None:
|
|
seg = await IdentitySegmentBuilder(tmp_path).build(_ctx(tmp_path))
|
|
legacy = ContextBuilder(workspace=tmp_path)._get_identity()
|
|
assert seg.text == legacy
|
|
|
|
async def test_identity_names_both_directories(self, tmp_path: Path) -> None:
|
|
"""The model is told where it works and where its memory lives, and the
|
|
two are not the same directory."""
|
|
from raven.agent.workdir import bind
|
|
|
|
home = tmp_path / "home"
|
|
home.mkdir()
|
|
project = tmp_path / "project"
|
|
project.mkdir()
|
|
|
|
with bind(project):
|
|
seg = await IdentitySegmentBuilder(home).build(_ctx(home))
|
|
|
|
assert f"Working directory: {project}" in seg.text
|
|
assert f"Agent home: {home}" in seg.text
|
|
assert f"{home}/user_memory/profile/user.md" in seg.text
|
|
assert str(project / "user_memory") not in seg.text
|
|
|
|
async def test_identity_without_a_backend_still_names_both_memory_files(self, tmp_path: Path) -> None:
|
|
"""No backend, no change.
|
|
|
|
Whether an episodic log nothing writes earns a line in the prompt is a
|
|
separate question from where memory lives once a backend owns it, and
|
|
deciding it here would tie the two together.
|
|
"""
|
|
seg = await IdentitySegmentBuilder(tmp_path).build(_ctx(tmp_path))
|
|
assert f"{tmp_path}/user_memory/profile/user.md" in seg.text
|
|
assert f"{tmp_path}/user_memory/episodic/episodes.md" in seg.text
|
|
|
|
async def test_identity_with_a_backend_names_no_profile_file(self, tmp_path: Path) -> None:
|
|
"""A wired backend owns memory, so agent home holds no file for it.
|
|
|
|
Told otherwise, the agent treats ``user.md`` as the place a
|
|
"remember this" goes. On a non-interactive turn that ``edit_file``
|
|
is refused at the ask tier, and the agent reports the refusal as the
|
|
memory system having failed -- while the backend has already stored
|
|
the turn and recalls it in the next session.
|
|
|
|
The same block forbids the claim in either direction. ``backend.store``
|
|
is dispatched after the turn ends, so neither "saved" nor "not saved"
|
|
is something this side watched happen -- and the claim is extracted
|
|
into memory next turn as if it were an observation of the system.
|
|
"""
|
|
seg = await IdentitySegmentBuilder(tmp_path, has_memory_backend=True).build(_ctx(tmp_path))
|
|
assert "profile/user.md" not in seg.text
|
|
assert "# Memory" in seg.text
|
|
assert "never report it to the user as done or as failed" in seg.text
|
|
|
|
async def test_identity_with_a_backend_names_the_log_read_only(self, tmp_path: Path) -> None:
|
|
"""The log is written by the host and worth a grep, so the agent is told it exists.
|
|
|
|
Read-only, in those words: the profile path stays out, because a
|
|
path under "your own memory" is what sent the agent to edit_file.
|
|
"""
|
|
seg = await IdentitySegmentBuilder(tmp_path, has_memory_backend=True).build(_ctx(tmp_path))
|
|
assert seg.text.count(f"{tmp_path}/user_memory/episodic/episodes.md") == 1
|
|
log_line = next(line for line in seg.text.splitlines() if "episodes.md" in line)
|
|
assert "read-only" in log_line
|
|
assert "profile/user.md" not in seg.text
|
|
assert "never report it to the user as done or as failed" in seg.text
|
|
|
|
async def test_identity_with_a_backend_matches_the_estimation_prompt(self, tmp_path: Path) -> None:
|
|
"""``_get_identity`` renders what the turn renders, backend included."""
|
|
seg = await IdentitySegmentBuilder(tmp_path, has_memory_backend=True).build(_ctx(tmp_path))
|
|
legacy = ContextBuilder(workspace=tmp_path, has_memory_backend=True)._get_identity()
|
|
assert seg.text == legacy
|
|
|
|
async def test_the_identity_renders_the_task_it_is_handed(self, tmp_path: Path) -> None:
|
|
"""The segment owns how the identity reads and nothing about deciding it.
|
|
|
|
It used to reach into the dispatch layer's ContextVar itself; Memory
|
|
fills the two strings now, so this hands them over the way the
|
|
assembler does and asserts only on the rendering.
|
|
"""
|
|
brief = "Only look at A. Leave B alone."
|
|
plain = (await IdentitySegmentBuilder(tmp_path).build(_ctx(tmp_path))).text
|
|
briefed = (
|
|
await IdentitySegmentBuilder(tmp_path).build(
|
|
_ctx(tmp_path, task_brief=brief, task_done_when="both tables land")
|
|
)
|
|
).text
|
|
|
|
assert "## This task" not in plain, "a turn with no brief reads as it always did"
|
|
assert plain in briefed.replace("\n\n## This task", ""), "the brief is appended, never substituted"
|
|
assert brief in briefed and "both tables land" in briefed
|
|
|
|
async def test_memory_is_what_fills_the_turn_from_the_charter(self, tmp_path: Path) -> None:
|
|
"""The other half of the split: the role that decides what a turn shows
|
|
its model is the one that reads the dispatch's charter."""
|
|
from raven.agent.harness.memory import DefaultMemory
|
|
from raven.agent.subagent.charter import Charter, charter_scope
|
|
from raven.contracts.context import TurnContext
|
|
|
|
brief = "Only look at A. Leave B alone."
|
|
plain = TurnContext(current_message="hi")
|
|
assert DefaultMemory._briefed(plain) is plain, "no charter bound, nothing to add"
|
|
|
|
with charter_scope(Charter(prompt=brief, stop_when="both tables land")):
|
|
briefed = DefaultMemory._briefed(plain)
|
|
|
|
assert briefed.task_brief == brief
|
|
assert briefed.task_done_when == "both tables land"
|
|
assert plain.task_brief == "", "the turn it was handed is not mutated"
|
|
|
|
async def test_identity_falls_back_to_agent_home_when_unbound(self, tmp_path: Path) -> None:
|
|
"""No binding means the pre-split single-directory behaviour."""
|
|
seg = await IdentitySegmentBuilder(tmp_path).build(_ctx(tmp_path))
|
|
assert f"Working directory: {tmp_path}" in seg.text
|
|
|
|
async def test_bootstrap_none_when_no_files(self, tmp_path: Path) -> None:
|
|
seg = await BootstrapSegmentBuilder(tmp_path).build(_ctx(tmp_path))
|
|
assert seg is None
|
|
|
|
async def test_bootstrap_renders_existing(self, tmp_path: Path) -> None:
|
|
(tmp_path / "TOOLS.md").write_text("tool docs", encoding="utf-8")
|
|
seg = await BootstrapSegmentBuilder(tmp_path).build(_ctx(tmp_path))
|
|
assert seg is not None
|
|
assert "## TOOLS.md" in seg.text
|
|
assert "tool docs" in seg.text
|
|
|
|
def test_identity_contains_model_id(self, tmp_path: Path) -> None:
|
|
prompt = render.identity_text(tmp_path, model="openrouter/some-model")
|
|
assert "openrouter/some-model" in prompt
|
|
|
|
def test_identity_installs_packages_into_a_virtual_environment(self, tmp_path: Path, monkeypatch) -> None:
|
|
"""Both platform policies carry the same install rule, inside the policy."""
|
|
import platform
|
|
|
|
sentence = "Do not install it into the global environment."
|
|
|
|
def policy(system: str) -> str:
|
|
monkeypatch.setattr(platform, "system", lambda: system)
|
|
text = render.identity_text(tmp_path, model="openrouter/some-model")
|
|
assert text.count(sentence) == 1
|
|
return text.split("## Platform Policy", 1)[1].split("## ", 1)[0]
|
|
|
|
for system in ("Darwin", "Windows"):
|
|
body = policy(system)
|
|
assert sentence in body, system
|
|
assert "this project's virtual environment" in body
|
|
assert "a temporary one" in body
|
|
assert "GNU tools" in policy("Windows")
|
|
assert "standard shell tools" in policy("Linux")
|
|
|
|
def test_identity_default_model_resolved_lazily(self, tmp_path: Path, monkeypatch) -> None:
|
|
monkeypatch.setattr(render, "_resolved_model_id", lambda: "openrouter/acme/lazy-model")
|
|
assert "openrouter/acme/lazy-model" in render.identity_text(tmp_path)
|
|
|
|
def test_legacy_identity_contains_model_id(self, tmp_path: Path, monkeypatch) -> None:
|
|
monkeypatch.setattr(render, "_resolved_model_id", lambda: "openrouter/acme/lazy-model")
|
|
legacy = ContextBuilder(workspace=tmp_path)._get_identity()
|
|
assert "openrouter/acme/lazy-model" in legacy
|
|
|
|
def test_resolved_model_id_codex_matches_wire_form(self, monkeypatch) -> None:
|
|
"""openai_codex bypasses LiteLLM and its client strips the provider
|
|
prefix before sending; the identity line must report that wire form,
|
|
not the stored one."""
|
|
cfg = _provider_config("openai-codex/gpt-5.1-codex", "openai_codex")
|
|
monkeypatch.setattr("raven.config.loader.load_config", lambda: cfg)
|
|
assert render._resolved_model_id() == "gpt-5.1-codex"
|
|
|
|
def test_resolved_model_id_azure_matches_wire_form(self, monkeypatch) -> None:
|
|
"""azure_openai sends the id as a URL deployment name with the prefix
|
|
stripped; the identity line must match."""
|
|
cfg = _provider_config("azure_openai/gpt-4o", "azure_openai")
|
|
monkeypatch.setattr("raven.config.loader.load_config", lambda: cfg)
|
|
assert render._resolved_model_id() == "gpt-4o"
|
|
|
|
def test_resolved_model_id_gateway_prefix_applied(self, monkeypatch) -> None:
|
|
cfg = _provider_config("acme/some-model", "openrouter", api_key="sk-or-v1-abc")
|
|
monkeypatch.setattr("raven.config.loader.load_config", lambda: cfg)
|
|
assert render._resolved_model_id() == "openrouter/acme/some-model"
|
|
|
|
|
|
class TestIdentityNamesTheBoundModel:
|
|
"""The identity line names the model the running turn goes out under.
|
|
|
|
The loop opens ``use_binding`` around every turn with the session's own
|
|
binding -- its ``/model`` pick, else the default -- so the renderer reads
|
|
the id there instead of from ``agents.defaults.model``. Before, the line
|
|
named the configured default on every turn of a switched conversation, and
|
|
the model, asked what it was, quoted the line back: from the page the
|
|
switch looked like it had never happened (#470).
|
|
"""
|
|
|
|
def test_a_switched_conversation_is_told_the_model_it_is_bound_to(self, tmp_path: Path, monkeypatch) -> None:
|
|
from raven.providers.binding import ModelBinding, use_binding
|
|
|
|
cfg = _provider_config("acme/config-default", "openrouter", api_key="sk-or-v1-abc")
|
|
monkeypatch.setattr("raven.config.loader.load_config", lambda: cfg)
|
|
|
|
with use_binding(ModelBinding(provider=object(), model="acme/session-pick")): # type: ignore[arg-type]
|
|
line = render._resolved_model_id()
|
|
|
|
assert line == "openrouter/acme/session-pick"
|
|
assert "config-default" not in line, "the configured default must not leak into a switched turn"
|
|
|
|
def test_the_bound_id_goes_through_the_same_storage_to_wire_conversion(self, tmp_path: Path, monkeypatch) -> None:
|
|
"""A session's pick is stored in the storage spelling, like the default.
|
|
Fed through the binding it must still come out in the wire spelling the
|
|
request carries -- here the codex client's, which strips the prefix --
|
|
or the line names an id no request ever sends."""
|
|
from raven.providers.binding import ModelBinding, use_binding
|
|
|
|
cfg = _provider_config("openai-codex/gpt-5.1", "openai_codex")
|
|
monkeypatch.setattr("raven.config.loader.load_config", lambda: cfg)
|
|
|
|
with use_binding(ModelBinding(provider=object(), model="openai-codex/gpt-5.1-codex")): # type: ignore[arg-type]
|
|
assert render._resolved_model_id() == "gpt-5.1-codex"
|
|
|
|
def test_outside_a_turn_the_configured_default_still_answers(self, monkeypatch) -> None:
|
|
from raven.providers.binding import active_binding
|
|
|
|
assert active_binding() is None
|
|
cfg = _provider_config("acme/config-default", "openrouter", api_key="sk-or-v1-abc")
|
|
monkeypatch.setattr("raven.config.loader.load_config", lambda: cfg)
|
|
|
|
assert render._resolved_model_id() == "openrouter/acme/config-default"
|
|
|
|
async def test_the_segment_names_the_bound_model(self, tmp_path: Path, monkeypatch) -> None:
|
|
from raven.providers.binding import ModelBinding, use_binding
|
|
|
|
cfg = _provider_config("acme/config-default", "openrouter", api_key="sk-or-v1-abc")
|
|
monkeypatch.setattr("raven.config.loader.load_config", lambda: cfg)
|
|
|
|
with use_binding(ModelBinding(provider=object(), model="acme/session-pick")): # type: ignore[arg-type]
|
|
seg = await IdentitySegmentBuilder(tmp_path).build(_ctx(tmp_path))
|
|
|
|
assert "You are running on model: openrouter/acme/session-pick." in seg.text
|
|
assert "config-default" not in seg.text
|
|
|
|
async def test_the_estimation_prompt_and_the_turn_prompt_agree_on_a_switched_model(
|
|
self, tmp_path: Path, monkeypatch
|
|
) -> None:
|
|
"""``ContextBuilder._get_identity`` sizes the system prompt for the token
|
|
budget and promises never to drift from the per-turn renderer. Both
|
|
read the binding, so a switched conversation is sized against the
|
|
line it is actually sent -- a field handed down the assembly path
|
|
would have reached one of the two and not the other."""
|
|
from raven.providers.binding import ModelBinding, use_binding
|
|
|
|
cfg = _provider_config("acme/config-default", "openrouter", api_key="sk-or-v1-abc")
|
|
monkeypatch.setattr("raven.config.loader.load_config", lambda: cfg)
|
|
|
|
with use_binding(ModelBinding(provider=object(), model="acme/session-pick")): # type: ignore[arg-type]
|
|
seg = await IdentitySegmentBuilder(tmp_path).build(_ctx(tmp_path))
|
|
estimate = ContextBuilder(workspace=tmp_path)._get_identity()
|
|
|
|
assert seg.text == estimate
|
|
assert "openrouter/acme/session-pick" in estimate
|
|
|
|
|
|
class TestMemory:
|
|
async def test_recall_merged_under_memory_heading(self, tmp_path: Path) -> None:
|
|
backend = _Backend([Memory(text="likes espresso")])
|
|
b = MemorySegmentBuilder(
|
|
ContextBuilder(workspace=tmp_path).memory,
|
|
backend=backend,
|
|
user_id="alice",
|
|
memory_top_k=7,
|
|
)
|
|
seg = await b.build(_ctx(tmp_path, "coffee"))
|
|
assert "# Memory" in seg.text
|
|
assert "- likes espresso" in seg.text
|
|
assert "# Recalled memory" not in seg.text
|
|
assert seg.meta["memory_hits"] == 1
|
|
assert backend.calls == [
|
|
{"query": "coffee", "user_id": "alice", "agent_id": None, "top_k": 7},
|
|
]
|
|
|
|
async def test_no_backend_empty_text(self, tmp_path: Path) -> None:
|
|
b = MemorySegmentBuilder(ContextBuilder(workspace=tmp_path).memory, backend=None)
|
|
seg = await b.build(_ctx(tmp_path))
|
|
# Empty workspace + no recall → no memory block.
|
|
assert seg.text == ""
|
|
assert seg.meta["memory_hits"] == 0
|
|
|
|
|
|
class TestRecallHasATurnBudget:
|
|
"""Memory is an enhancement, not a precondition for answering.
|
|
|
|
The recall was awaited with no bound at all, in front of the model call --
|
|
a backend that hung left the turn with nothing to show and no way out but
|
|
the user cancelling it.
|
|
"""
|
|
|
|
async def test_a_hanging_backend_degrades_to_no_hits(self, monkeypatch, tmp_path) -> None:
|
|
import asyncio
|
|
|
|
from raven.context_engine.segments import memory as memory_segment
|
|
|
|
monkeypatch.setattr(memory_segment, "_RECALL_BUDGET_S", 0.05)
|
|
|
|
class _Hanging:
|
|
async def recall(self, query, *, user_id=None, agent_id=None, top_k):
|
|
await asyncio.sleep(30)
|
|
|
|
builder = memory_segment.MemorySegmentBuilder(
|
|
ContextBuilder(workspace=tmp_path).memory,
|
|
backend=_Hanging(),
|
|
)
|
|
assert await builder._recall("hi") == []
|
|
|
|
async def test_a_raising_backend_degrades_to_no_hits(self, tmp_path) -> None:
|
|
from raven.context_engine.segments import memory as memory_segment
|
|
|
|
class _Broken:
|
|
async def recall(self, query, *, user_id=None, agent_id=None, top_k):
|
|
raise RuntimeError("memory service down")
|
|
|
|
builder = memory_segment.MemorySegmentBuilder(
|
|
ContextBuilder(workspace=tmp_path).memory,
|
|
backend=_Broken(),
|
|
)
|
|
assert await builder._recall("hi") == []
|
|
|
|
async def test_a_timeout_and_a_programming_error_log_distinguishably(self, monkeypatch, tmp_path, caplog) -> None:
|
|
"""A permanent bug (a mismatched ``recall`` signature, an
|
|
``AttributeError`` in a plugin) must not read the same as a slow-but-
|
|
healthy service: only a timeout should say "budget", and only the
|
|
other should name the exception that actually happened.
|
|
"""
|
|
import asyncio
|
|
import logging
|
|
|
|
from loguru import logger
|
|
|
|
from raven.context_engine.segments import memory as memory_segment
|
|
|
|
monkeypatch.setattr(memory_segment, "_RECALL_BUDGET_S", 0.05)
|
|
|
|
class _Hanging:
|
|
async def recall(self, query, *, user_id=None, agent_id=None, top_k):
|
|
await asyncio.sleep(30)
|
|
|
|
class _WrongSignature:
|
|
async def recall(self, query, *, user_id=None, agent_id=None, top_k):
|
|
raise TypeError("recall() got an unexpected keyword argument 'foo'")
|
|
|
|
# Bridge loguru -> stdlib caplog (loguru doesn't write to logging by default)
|
|
handler_id = logger.add(lambda msg: logging.getLogger("loguru.bridge").warning(msg), level="WARNING")
|
|
try:
|
|
with caplog.at_level(logging.WARNING, logger="loguru.bridge"):
|
|
timeout_builder = memory_segment.MemorySegmentBuilder(
|
|
ContextBuilder(workspace=tmp_path).memory,
|
|
backend=_Hanging(),
|
|
)
|
|
await timeout_builder._recall("hi")
|
|
timeout_text = caplog.records[-1].message
|
|
caplog.clear()
|
|
|
|
error_builder = memory_segment.MemorySegmentBuilder(
|
|
ContextBuilder(workspace=tmp_path).memory,
|
|
backend=_WrongSignature(),
|
|
)
|
|
await error_builder._recall("hi")
|
|
error_text = caplog.records[-1].message
|
|
finally:
|
|
logger.remove(handler_id)
|
|
|
|
assert timeout_text != error_text
|
|
assert "budget" in timeout_text
|
|
assert "TypeError" in error_text
|
|
assert "unexpected keyword argument" in error_text
|
|
|
|
async def test_a_healthy_backend_is_untouched(self, tmp_path) -> None:
|
|
from raven.context_engine.segments import memory as memory_segment
|
|
|
|
class _Fine:
|
|
async def recall(self, query, *, user_id=None, agent_id=None, top_k):
|
|
return ["hit"]
|
|
|
|
builder = memory_segment.MemorySegmentBuilder(
|
|
ContextBuilder(workspace=tmp_path).memory,
|
|
backend=_Fine(),
|
|
)
|
|
assert await builder._recall("hi") == ["hit"]
|
|
|
|
def test_the_plugin_budget_is_strictly_inside_the_framework_one(self) -> None:
|
|
"""Equal budgets would let the framework's cancellation pre-empt the
|
|
plugin's own timeout handling, which is what demotes the service and
|
|
makes every later turn cost nothing."""
|
|
from raven.context_engine.segments import memory as memory_segment
|
|
from raven_everos import backend as everos_backend
|
|
|
|
assert everos_backend._RECALL_TIMEOUT_S < memory_segment._RECALL_BUDGET_S
|
|
|
|
|
|
class TestSkills:
|
|
async def test_router_hits_render_into_skills(self, tmp_path: Path) -> None:
|
|
hits = [RouterHit(qualified_id="local/g", name="g", content="how to git", score=0.9)]
|
|
b = SkillsSegmentBuilder(SkillForgeRouter([_Source(hits)]), skill_top_k=5)
|
|
seg = await b.build(_ctx(tmp_path))
|
|
assert seg.text.startswith("# Skills")
|
|
assert "### Skill: g [local/g]" in seg.text
|
|
assert "how to git" in seg.text
|
|
assert seg.meta["injected_skill_ids"] == ["local/g"]
|
|
|
|
async def test_empty_hits_empty_text(self, tmp_path: Path) -> None:
|
|
b = SkillsSegmentBuilder(SkillForgeRouter([]), skill_top_k=5)
|
|
seg = await b.build(_ctx(tmp_path))
|
|
assert seg.text == ""
|
|
assert seg.meta["injected_skill_ids"] == []
|
|
|
|
|
|
class TestCollectToolNames:
|
|
"""Shared by segments 4 and 5, with opposite consequences for a wrong
|
|
answer: segment 5 loses a gate hint, segment 4 loses content. Both read
|
|
``None`` as 'do not gate', so the empty-vs-unknown distinction is load
|
|
bearing and pinned here."""
|
|
|
|
def test_reads_openai_and_flat_shapes(self) -> None:
|
|
got = render.collect_tool_names(lambda: [{"function": {"name": "a"}}, {"name": "b"}])
|
|
assert got == ["a", "b"]
|
|
|
|
def test_unwired_and_raising_and_empty_are_all_none(self) -> None:
|
|
def _boom():
|
|
raise RuntimeError("x")
|
|
|
|
assert render.collect_tool_names(None) is None
|
|
assert render.collect_tool_names(_boom) is None
|
|
# Empty list collapses to None too -- not because zero tools is
|
|
# unreachable (naming every registered tool in tools.disabled_tools
|
|
# reaches it) but because this helper's consumer has no always-skill to
|
|
# withhold when the agent holds nothing. A caller that must tell the two
|
|
# apart reads _tool_names; see live_dispatch_tools.
|
|
assert render.collect_tool_names(lambda: []) is None
|
|
|
|
def test_malformed_entries_are_skipped_not_fatal(self) -> None:
|
|
got = render.collect_tool_names(lambda: ["junk", {"function": "notadict"}, {"name": "ok"}])
|
|
assert got == ["ok"]
|
|
|
|
|
|
class TestActiveSkills:
|
|
async def test_none_on_empty_workspace(self, tmp_path: Path) -> None:
|
|
b = ActiveSkillsSegmentBuilder(ContextBuilder(workspace=tmp_path).skills)
|
|
seg = await b.build(_ctx(tmp_path))
|
|
# Built-in always-skills may exist; assert the builder either skips
|
|
# or emits a well-formed # Active Skills block (never malformed).
|
|
if seg is not None:
|
|
assert seg.text.startswith("# Active Skills")
|
|
|
|
async def test_dag_skill_is_resident_as_a_digest(self, tmp_path: Path) -> None:
|
|
"""With its tool registered, the shipped orchestration skill reaches the
|
|
system prompt every turn — as description + routes only, body on disk."""
|
|
b = ActiveSkillsSegmentBuilder(
|
|
ContextBuilder(workspace=tmp_path).skills,
|
|
get_tool_definitions=_tool_defs("read_file", "run_subagent_dag"),
|
|
)
|
|
seg = await b.build(_ctx(tmp_path))
|
|
|
|
assert seg is not None
|
|
assert "### Skill: subagent-dag-orchestration" in seg.text
|
|
assert "run_subagent_dag" in seg.text # from the description
|
|
assert 'read_skill("local/subagent-dag-orchestration")' in seg.text
|
|
# A distinctive line from deep in SKILL.md, i.e. the body proper.
|
|
assert "## When to use" not in seg.text
|
|
|
|
async def test_skill_withheld_when_its_required_tool_is_absent(self, tmp_path: Path) -> None:
|
|
"""``run_subagent_dag`` only registers when third-party sub-agents are
|
|
configured, but the skill advertising it ships always-on. Without this
|
|
gate the agent is told every turn to reach for a tool it cannot call."""
|
|
b = ActiveSkillsSegmentBuilder(
|
|
ContextBuilder(workspace=tmp_path).skills,
|
|
get_tool_definitions=_tool_defs("read_file", "spawn", "exec"),
|
|
)
|
|
seg = await b.build(_ctx(tmp_path))
|
|
|
|
assert seg is None or "subagent-dag-orchestration" not in seg.text
|
|
|
|
async def test_coding_skill_is_resident_as_a_digest_beside_spawn(self, tmp_path: Path) -> None:
|
|
"""The multi-agent coding skill rides on ``spawn`` alone: a turn that can
|
|
hand code to a coding agent carries its description and route every turn,
|
|
and the body stays on disk until the agent reads it."""
|
|
b = ActiveSkillsSegmentBuilder(
|
|
ContextBuilder(workspace=tmp_path).skills,
|
|
get_tool_definitions=_tool_defs("read_file", "spawn", "exec"),
|
|
)
|
|
seg = await b.build(_ctx(tmp_path))
|
|
|
|
assert seg is not None
|
|
assert "### Skill: multi-agent-coding" in seg.text
|
|
assert 'read_skill("local/multi-agent-coding")' in seg.text
|
|
assert "Create a git worktree of your own" not in seg.text
|
|
|
|
async def test_coding_skill_withheld_when_spawn_is(self, tmp_path: Path) -> None:
|
|
"""Its procedure continues an instance with ``spawn``'s own fields, so a
|
|
turn whose ``spawn`` is disabled must not be told to follow it, whatever
|
|
else it can dispatch with."""
|
|
b = ActiveSkillsSegmentBuilder(
|
|
ContextBuilder(workspace=tmp_path).skills,
|
|
get_tool_definitions=_tool_defs("read_file", "run_subagent_dag", "exec"),
|
|
)
|
|
seg = await b.build(_ctx(tmp_path))
|
|
|
|
assert seg is None or "multi-agent-coding" not in seg.text
|
|
|
|
def test_coding_skill_picks_a_worker_label_when_a_table_is_bound(self, tmp_path: Path) -> None:
|
|
"""On such a turn `subagent` takes labels alone, so a coding agent picked from the
|
|
roster -- first pick or a quota takeover -- is a value the enum refuses."""
|
|
body = " ".join(ContextBuilder(workspace=tmp_path).skills.load_skill("multi-agent-coding").split())
|
|
assert "pick the worker whose brief covers the coding work" in body
|
|
assert "another worker whose brief covers it and that runs on a different agent" in body
|
|
assert "outside the enum" in body
|
|
|
|
async def test_unwired_tool_lookup_does_not_gate(self, tmp_path: Path) -> None:
|
|
"""No callable wired → unknown, not empty. A wiring gap must degrade to
|
|
showing the skill, never to silently blanking the segment."""
|
|
b = ActiveSkillsSegmentBuilder(ContextBuilder(workspace=tmp_path).skills)
|
|
seg = await b.build(_ctx(tmp_path))
|
|
|
|
assert seg is not None
|
|
assert "subagent-dag-orchestration" in seg.text
|
|
|
|
async def test_malformed_requires_does_not_break_or_hide_a_skill(self, tmp_path: Path) -> None:
|
|
"""A hand-authored ``requires`` of the wrong shape must not raise into
|
|
prompt assembly, nor silently withhold the skill."""
|
|
skill_dir = tmp_path / "skills" / "wonky"
|
|
skill_dir.mkdir(parents=True)
|
|
skill_dir.joinpath("SKILL.md").write_text(
|
|
'---\nname: wonky\ndescription: d\nmetadata: {"raven":{"always":true,"requires":{"tools":7}}}\n---\n\nbody\n',
|
|
encoding="utf-8",
|
|
)
|
|
b = ActiveSkillsSegmentBuilder(
|
|
ContextBuilder(workspace=tmp_path).skills,
|
|
get_tool_definitions=_tool_defs("read_file"),
|
|
)
|
|
seg = await b.build(_ctx(tmp_path))
|
|
|
|
assert seg is not None
|
|
assert "### Skill: wonky" in seg.text
|
|
|
|
async def test_raising_tool_lookup_does_not_gate(self, tmp_path: Path) -> None:
|
|
def _boom() -> list[dict]:
|
|
raise RuntimeError("registry unavailable")
|
|
|
|
b = ActiveSkillsSegmentBuilder(
|
|
ContextBuilder(workspace=tmp_path).skills,
|
|
get_tool_definitions=_boom,
|
|
)
|
|
seg = await b.build(_ctx(tmp_path))
|
|
|
|
assert seg is not None
|
|
assert "subagent-dag-orchestration" in seg.text
|
|
|
|
|
|
class TestIdentityDelegationSection:
|
|
"""A sub-agent that declares what it owns gets a line in the prompt telling
|
|
the model not to do that work itself. Only declared agents appear, so an
|
|
install with none reads exactly as it did before."""
|
|
|
|
@staticmethod
|
|
def _meta(name: str, owns: str = ""):
|
|
from raven.agent.subagent.backends import AgentMeta
|
|
|
|
return AgentMeta(name, f"{name} description", True, True, False, owns)
|
|
|
|
def test_declared_agents_get_an_ownership_line(self, tmp_path: Path) -> None:
|
|
text = render.identity_text(
|
|
tmp_path,
|
|
specialists=[("Scribe", "owns decks. Do not build the deck yourself.")],
|
|
)
|
|
assert "## Delegation" in text
|
|
assert "`Scribe` owns decks. Do not build the deck yourself." in text
|
|
|
|
def test_the_section_is_the_ownership_list_alone(self, tmp_path: Path) -> None:
|
|
"""How work is handed over, and what the host's own tools are for, is the
|
|
``## Your role`` section's to say. Said again here, it told the model its own tools were
|
|
for "work none of them covers" -- the work the role sends to the generic
|
|
agent instead."""
|
|
text = render.identity_text(tmp_path, specialists=[("Scribe", "owns decks.")])
|
|
delegation = text.split("## Delegation", 1)[1].split("\n## ", 1)[0]
|
|
assert "Hand the whole task over" not in delegation
|
|
assert "Your own tools are for work" not in text
|
|
assert "You are an orchestrator first" not in text
|
|
|
|
def test_nothing_declared_keeps_the_joint_the_section_took_over(self, tmp_path: Path) -> None:
|
|
"""An install with no declaring agent must read exactly as it did before.
|
|
|
|
This test used to compare ``identity_text`` against itself -- ``specialists=[]``
|
|
against the default -- which agrees whatever the output is. It passed while
|
|
the section had moved the blank line before ``## Raven Guidelines`` inside
|
|
itself and dropped it on the empty branch, so the prompt every install
|
|
without a specialist renders was one blank line short. The joint is the
|
|
thing to assert; the self-comparison is kept only as the weaker half.
|
|
"""
|
|
plain = render.identity_text(tmp_path, specialists=[])
|
|
assert plain == render.identity_text(tmp_path)
|
|
assert "## Delegation" not in plain
|
|
# Three newlines, not two: the platform policy block ends with one of its
|
|
# own, so asserting two passes with the separator dropped as well.
|
|
assert "\n\n\n## Raven Guidelines" in plain
|
|
|
|
def test_the_section_sits_above_the_guidelines(self, tmp_path: Path) -> None:
|
|
"""A rule about which agent to name has to be read before the general
|
|
guidelines it qualifies, not after them."""
|
|
text = render.identity_text(tmp_path, specialists=[("Scribe", "owns decks.")])
|
|
assert text.index("## Delegation") < text.index("## Raven Guidelines")
|
|
|
|
async def test_builder_collects_only_agents_that_declare_ownership(self, tmp_path: Path) -> None:
|
|
roster = [self._meta("Scribe", "owns decks."), self._meta("Nomad")]
|
|
builder = IdentitySegmentBuilder(tmp_path, list_subagents=lambda: roster)
|
|
text = (await builder.build(_ctx(tmp_path))).text
|
|
assert "`Scribe` owns decks." in text
|
|
assert "Nomad" not in text
|
|
|
|
async def test_builder_survives_a_roster_lookup_that_raises(self, tmp_path: Path) -> None:
|
|
"""Prompt assembly must not fail because the agent table is mid-rebuild."""
|
|
|
|
def _boom():
|
|
raise RuntimeError("table is being rebuilt")
|
|
|
|
builder = IdentitySegmentBuilder(tmp_path, list_subagents=_boom)
|
|
assert "## Delegation" not in (await builder.build(_ctx(tmp_path))).text
|
|
|
|
async def test_the_generic_row_is_never_a_specialist(self, tmp_path: Path) -> None:
|
|
"""It carries no capability bias, so it owns no kind of work -- and a row
|
|
that declared some would render a prohibition aimed at the agent reading
|
|
it."""
|
|
builder = IdentitySegmentBuilder(
|
|
tmp_path, list_subagents=lambda: [self._meta(GENERIC_AGENT, "owns everything.")]
|
|
)
|
|
assert "## Delegation" not in (await builder.build(_ctx(tmp_path))).text
|
|
|
|
|
|
def _role_of(text: str) -> str:
|
|
assert "## Your role" in text, "baseline: the ``## Your role`` section must be present, or this asserts nothing"
|
|
return text.split("## Your role", 1)[1].split("\n## ", 1)[0]
|
|
|
|
|
|
class TestHostRole:
|
|
"""The host is told what it is for: running the user's work through the
|
|
roster, not doing that work itself.
|
|
|
|
Replayed on a recorded conversation (glm-5.3-flash): with this section, a
|
|
question about a dispatched task's work went back to that task's instance in
|
|
6 of 6 runs against 0 of 6 without it, and a small script went to an agent in
|
|
6 of 6 instead of being written by the host in 3 of 3. Naming the host
|
|
differently without the section moved nothing (0 of 6).
|
|
"""
|
|
|
|
def test_the_host_is_the_users_agent_manager(self, tmp_path: Path) -> None:
|
|
text = render.identity_text(tmp_path)
|
|
assert "You are Raven, the user's agent manager." in text
|
|
assert "a helpful AI assistant" not in text
|
|
assert text.index("## Your role") < text.index("## Runtime"), (
|
|
"the role decides how every request is read, so it is read before anything else"
|
|
)
|
|
|
|
async def test_resident_dag_guidance_leaves_delegation_to_the_host_role(self, tmp_path: Path) -> None:
|
|
tools = _tool_defs("read_file", "spawn", "run_subagent_dag", "exec")
|
|
catalog = ContextBuilder(workspace=tmp_path).skills
|
|
identity = await IdentitySegmentBuilder(tmp_path, get_tool_definitions=tools).build(_ctx(tmp_path))
|
|
skills = await ActiveSkillsSegmentBuilder(catalog, get_tool_definitions=tools).build(_ctx(tmp_path))
|
|
body = catalog.load_skill("subagent-dag-orchestration")
|
|
|
|
assert "Producing anything, however small" in identity.text
|
|
assert "goes to an agent" in identity.text
|
|
assert "### Skill: subagent-dag-orchestration" in skills.text
|
|
for text in (skills.text, body):
|
|
normalized = " ".join(text.split())
|
|
assert "The host's role decides what to delegate" in normalized
|
|
assert "`subagent` parameter enum" in normalized
|
|
assert "worker labels" in normalized
|
|
assert "do those yourself" not in normalized
|
|
assert "work that never cleared the bound" not in normalized
|
|
assert "nodes reach third-party" not in normalized
|
|
assert "one-node graph" in body
|
|
|
|
def test_without_a_dispatch_path_the_host_does_the_work_itself(self, tmp_path: Path) -> None:
|
|
"""A role that hands all work over is a prohibition without a way to obey it
|
|
on a turn that holds no dispatch tool -- the same guard the Delegation
|
|
section keeps."""
|
|
text = render.identity_text(tmp_path, dispatch_tools=())
|
|
assert "You are Raven, a helpful AI assistant." in text
|
|
assert "## Your role" not in text
|
|
assert "agent manager" not in text
|
|
|
|
def test_the_role_names_only_the_paths_the_turn_holds(self, tmp_path: Path) -> None:
|
|
spawn_only = _role_of(render.identity_text(tmp_path, dispatch_tools=("spawn",)))
|
|
assert "`spawn`" in spawn_only
|
|
assert "run_subagent_dag" not in spawn_only
|
|
|
|
dag_only = _role_of(render.identity_text(tmp_path, dispatch_tools=("run_subagent_dag",)))
|
|
assert "`run_subagent_dag`" in dag_only
|
|
assert "`spawn`" not in dag_only
|
|
|
|
def test_work_no_agent_fits_goes_to_the_generic_agent(self, tmp_path: Path) -> None:
|
|
"""Fit comes first, by ownership or by description. Named bare as where
|
|
unowned work goes, the generic agent took a small script in 4 of 6 replayed
|
|
runs on an install whose coding agents declare no ownership."""
|
|
role = _role_of(render.identity_text(tmp_path))
|
|
assert f"and to `{GENERIC_AGENT}` only when no agent's does" in role
|
|
assert "whose ownership or description fits it" in role
|
|
|
|
def test_a_question_about_a_tasks_work_is_its_agents_to_answer(self, tmp_path: Path) -> None:
|
|
"""The recorded failure was a question -- "why is it hardcoded?" -- answered by
|
|
the host from its own reading of the code. Only a role that names questions
|
|
among the follow-ups moved it."""
|
|
role = _role_of(render.identity_text(tmp_path))
|
|
assert "to that task's instance" in role
|
|
assert "a question about how or why it was done" in role
|
|
|
|
def test_the_production_rule_has_no_size_exception(self, tmp_path: Path) -> None:
|
|
"""An earlier wording let the host answer "quick questions" itself, and the
|
|
model stretched that into "small scripts I can do directly": 0 of 4 runs
|
|
delegated one. With "however small" it was 6 of 6."""
|
|
role = _role_of(render.identity_text(tmp_path))
|
|
assert "however small" in role
|
|
assert "quick question" not in role
|
|
|
|
def test_on_a_worker_table_work_goes_to_a_worker_by_label(self, tmp_path: Path) -> None:
|
|
"""A bound table narrows `subagent` to its labels, and a generated Persona never
|
|
offers the generic agent as a worker. The role must not name a value the enums
|
|
refuse, nor present the agents behind the labels as values."""
|
|
role = _role_of(render.identity_text(tmp_path, worker_table=True))
|
|
assert GENERIC_AGENT not in role
|
|
assert "whose ownership or description fits it" not in role
|
|
assert "named by its label from the `subagent` enum" in role
|
|
assert "they are not values to pass" in role
|
|
assert "however small" in role
|
|
|
|
def test_work_no_worker_covers_is_put_to_the_user(self, tmp_path: Path) -> None:
|
|
"""Not done by the host, which the role forbids, and not sent outside the table,
|
|
which the enums refuse."""
|
|
role = _role_of(render.identity_text(tmp_path, worker_table=True))
|
|
assert "When no worker's brief covers a piece of work, do not do it yourself" in role
|
|
assert "tell the user which work no worker covers" in role
|
|
assert "dispatch it only once they have answered" in role
|
|
|
|
def test_a_worker_table_moves_only_the_production_rule(self, tmp_path: Path) -> None:
|
|
"""The bullets were measured on replays; only the paragraph that says where work goes moves."""
|
|
plain = _role_of(render.identity_text(tmp_path))
|
|
bound = _role_of(render.identity_text(tmp_path, worker_table=True))
|
|
assert plain.split("\n\n", 1)[0] == bound.split("\n\n", 1)[0]
|
|
assert plain != bound
|
|
|
|
def test_a_worker_table_role_names_only_the_paths_the_turn_holds(self, tmp_path: Path) -> None:
|
|
dag_only = _role_of(render.identity_text(tmp_path, dispatch_tools=("run_subagent_dag",), worker_table=True))
|
|
assert "`run_subagent_dag`" in dag_only
|
|
assert "`spawn`" not in dag_only
|
|
|
|
def test_a_worker_table_adds_no_role_where_there_is_none(self, tmp_path: Path, monkeypatch) -> None:
|
|
"""No dispatch path, or a process launched to do one task: nothing is routed, so
|
|
the identity those had stays byte for byte."""
|
|
from raven.agent.subagent.role import SUBAGENT_ENV_VAR
|
|
|
|
assert render.identity_text(tmp_path, dispatch_tools=(), worker_table=True) == render.identity_text(
|
|
tmp_path, dispatch_tools=()
|
|
)
|
|
monkeypatch.setenv(SUBAGENT_ENV_VAR, "1")
|
|
assert render.identity_text(tmp_path, worker_table=True) == render.identity_text(tmp_path)
|
|
|
|
async def test_the_builder_reads_the_table_bound_for_the_turn(self, tmp_path: Path) -> None:
|
|
"""Read as the turn is assembled: an empty table and a closed scope read as no table."""
|
|
from raven.agent.subagent.delegate import DelegateTable, Worker, delegate_scope
|
|
|
|
builder = IdentitySegmentBuilder(tmp_path, get_tool_definitions=_tool_defs("spawn", "run_subagent_dag"))
|
|
plain = (await builder.build(_ctx(tmp_path))).text
|
|
table = DelegateTable(workers={"research-a": Worker("research-a", "Raven-Research", "competitor A's pricing")})
|
|
with delegate_scope(table):
|
|
bound = (await builder.build(_ctx(tmp_path))).text
|
|
with delegate_scope(DelegateTable(workers={})):
|
|
empty = (await builder.build(_ctx(tmp_path))).text
|
|
|
|
assert f"`{GENERIC_AGENT}` only when no agent's does" in _role_of(plain)
|
|
assert "named by its label from the `subagent` enum" in _role_of(bound)
|
|
assert empty == plain
|
|
assert (await builder.build(_ctx(tmp_path))).text == plain
|
|
|
|
def test_the_role_names_no_label_so_the_identity_stays_stable(self, tmp_path: Path) -> None:
|
|
"""A Worker Table generated in Task mode can change its labels every turn, and the
|
|
identity is a stable segment: the role says how to route and leaves the labels to the enum."""
|
|
from raven.agent.subagent.delegate import DelegateTable, Worker, delegate_scope
|
|
|
|
texts = []
|
|
for label in ("research-a", "writer-b"):
|
|
with delegate_scope(DelegateTable(workers={label: Worker(label, "Raven-Research", "one job")})):
|
|
texts.append(render.identity_text(tmp_path))
|
|
assert "named by its label from the `subagent` enum" in _role_of(texts[0])
|
|
assert texts[0] == texts[1]
|
|
assert "research-a" not in texts[0]
|
|
|
|
def test_a_table_bound_mid_turn_gets_the_routing_a_bound_turn_has(self, tmp_path: Path) -> None:
|
|
"""``load_playbook`` binds a table after the identity was assembled, so the routing a
|
|
table-bound role carries travels on that call's result instead, in the same words."""
|
|
note = render.worker_table_note(("spawn", "run_subagent_dag"))
|
|
role = _role_of(render.identity_text(tmp_path, worker_table=True))
|
|
assert note.startswith("\n\nNote: ")
|
|
assert "takes its worker labels, not agent names" in note
|
|
for rule in (
|
|
"named by its label from the `subagent` enum",
|
|
"When no worker's brief covers a piece of work, do not do it yourself",
|
|
):
|
|
assert rule in note
|
|
assert rule in role
|
|
|
|
def test_the_mid_turn_note_is_absent_where_no_role_routes(self, monkeypatch) -> None:
|
|
from raven.agent.subagent.role import SUBAGENT_ENV_VAR
|
|
|
|
assert render.worker_table_note(()) == ""
|
|
assert "named by its label" in render.worker_table_note(("run_subagent_dag",))
|
|
monkeypatch.setenv(SUBAGENT_ENV_VAR, "1")
|
|
assert render.worker_table_note(("spawn",)) == ""
|
|
|
|
|
|
class TestDelegationNeedsSomewhereToDelegateTo:
|
|
"""The section prohibits a specialist's kind of work, so it must not outlive
|
|
the tools that carry the work away. With every dispatch path withheld a
|
|
request has no compliant action left: the model either does the forbidden
|
|
thing or abandons the task."""
|
|
|
|
SPEC = [("Scribe", "owns decks. Do not build the deck yourself.")]
|
|
|
|
@staticmethod
|
|
def _defs(*names: str) -> list[dict]:
|
|
return [{"function": {"name": n}} for n in names]
|
|
|
|
def test_no_live_path_reads_like_an_install_without_specialists(self, tmp_path: Path) -> None:
|
|
assert render.identity_text(tmp_path, specialists=self.SPEC, dispatch_tools=()) == render.identity_text(
|
|
tmp_path, dispatch_tools=()
|
|
)
|
|
|
|
def test_only_the_paths_that_exist_are_named(self, tmp_path: Path) -> None:
|
|
spawn_only = render.identity_text(tmp_path, specialists=self.SPEC, dispatch_tools=("spawn",))
|
|
assert "`Scribe` owns decks." in spawn_only
|
|
assert "`spawn`" in spawn_only
|
|
assert "run_subagent_dag" not in spawn_only
|
|
|
|
def test_the_guard_is_not_too_tight_either(self, tmp_path: Path) -> None:
|
|
"""``spawn`` withheld while the graph tool is live is still a delegable
|
|
install -- suppressing the section there would lose a rule that applies."""
|
|
dag_only = render.identity_text(tmp_path, specialists=self.SPEC, dispatch_tools=("run_subagent_dag",))
|
|
assert "`Scribe` owns decks." in dag_only
|
|
assert "`run_subagent_dag`" in dag_only
|
|
assert "`spawn`" not in dag_only
|
|
|
|
def test_an_unknowable_tool_surface_does_not_gate(self) -> None:
|
|
"""A wiring gap must degrade to saying too much: the alternative deletes
|
|
the rule on every install that never wired the lookup up."""
|
|
|
|
def _boom():
|
|
raise RuntimeError("registry mid-rebuild")
|
|
|
|
assert render.live_dispatch_tools(None) == render.DISPATCH_TOOLS
|
|
assert render.live_dispatch_tools(_boom) == render.DISPATCH_TOOLS
|
|
|
|
def test_a_readable_but_empty_tool_table_does_gate(self) -> None:
|
|
"""This case was asserted the other way round when the gate was added,
|
|
which is how it shipped broken. An empty table is not an unreadable one:
|
|
it is a turn that can delegate nothing, reachable by naming every
|
|
registered tool in ``tools.disabledTools`` -- and a request needing no
|
|
tools at all is exactly the one the prohibition would then strand.
|
|
|
|
``collect_tool_names`` still folds empty into ``None`` for the
|
|
always-skills filter, so both halves of the divergence are pinned here:
|
|
it is a deliberate difference between two questions, not a drift.
|
|
"""
|
|
assert render.live_dispatch_tools(lambda: []) == ()
|
|
assert render.collect_tool_names(lambda: []) is None
|
|
|
|
def test_a_tool_list_without_any_dispatch_path_gates(self) -> None:
|
|
assert render.live_dispatch_tools(lambda: self._defs("grep", "read_file")) == ()
|
|
|
|
async def test_builder_drops_the_section_when_the_turn_offers_no_dispatch_tool(self, tmp_path: Path) -> None:
|
|
from raven.agent.subagent.backends import AgentMeta
|
|
|
|
builder = IdentitySegmentBuilder(
|
|
tmp_path,
|
|
list_subagents=lambda: [AgentMeta("Scribe", "d", True, True, False, "owns decks.")],
|
|
get_tool_definitions=lambda: self._defs("grep"),
|
|
)
|
|
assert "## Delegation" not in (await builder.build(_ctx(tmp_path))).text
|
|
|
|
async def test_builder_keeps_it_when_the_turn_offers_one(self, tmp_path: Path) -> None:
|
|
from raven.agent.subagent.backends import AgentMeta
|
|
|
|
builder = IdentitySegmentBuilder(
|
|
tmp_path,
|
|
list_subagents=lambda: [AgentMeta("Scribe", "d", True, True, False, "owns decks.")],
|
|
get_tool_definitions=lambda: self._defs("spawn"),
|
|
)
|
|
text = (await builder.build(_ctx(tmp_path))).text
|
|
assert "`Scribe` owns decks." in text
|
|
assert "run_subagent_dag" not in text
|