Raven-1/tests/test_update_subagents.py
Dizhan Xue 14edd7bdb2
feat(*): run the host as an agent manager that hands work, coding included, to sub-agents (#1010)
## Summary

Raven's host acts as the user's agent manager: it delegates production
work, including small edits, to the agent whose ownership or description
fits the task -- or, on a turn bound to a Worker Table, to the worker
whose brief covers it -- keeps track of the task and its instance,
routes follow-ups back to that instance, and gathers the results for the
user. Its own tools support briefing, checking and coordination.

- Add the resident `multi-agent-coding` skill for choosing coding
agents, keeping one instance per task, passing results by file,
isolating concurrent edits in worktrees, and handing over work when an
agent exhausts its quota.
- Keep the new host role authoritative in both the DAG skill's resident
description and its full guide. A single task uses `spawn`, or a
one-node graph when only the DAG dispatch path is available.
- Rename the generic sub-agent to `Raven-General`, with `Raven` and
`raven` retained as compatibility aliases. Instance records, handle
locks, live activity, cancellation, preferences and RPC lookups use the
same canonical identity. Historical spawn and DAG readers normalize
names before liveness checks and serialization, so tasks and transcripts
join the current instance roster. Concurrent alias calls preserve both
turns instead of overwriting one conversation snapshot. The instance
registry reads former-name rows under the current name and, on its next
write, saves every loaded row under it; transcripts, direct chats and
stored graphs stay as written.
- Resolve built-in configuration aliases before merging fields:
`Raven-General` wins over `Raven`, then `raven`, regardless of file
order. Discard older rows wholesale with a warning, preventing stale
model, permission or transport settings from leaking into the selected
row.
- Resume the most recently written message state and instance log across
former names. Saved messages take precedence over an empty chat
directory; equal timestamps prefer the current name and then the most
recent former name. Existing files stay in place. Legacy spawn records,
completed DAG manifests and unfinished graphs remain readable through
the current name.
- Emit the agent roster once at the end of `spawn`'s description. The
registry places `run_subagent_dag` and then `spawn` last, including
after late MCP or session tools. The graph tool renders the roster
itself when `spawn` is unavailable. With a Worker Table bound, dispatch
guidance selects worker labels from the live enum; the roster explains
underlying capabilities. The tool descriptions and resident DAG guide
follow the same rule.
- On a turn bound to a Worker Table, such as a Persona session, `spawn`
and the DAG nodes offer worker labels alone, and a generated Persona
never includes the generic agent. The host role on such a turn routes
production work to the worker whose brief covers it, by label, and puts
work no worker covers to the user, who can add a worker or give the
conversation another Harness, instead of doing it itself or naming an
agent outside the enum. Ordinary turns render byte for byte as before. A
Harness that `load_playbook` binds mid-turn takes effect after the
identity was assembled, so that call's result carries the same routing
as this system's trusted note, after the untrusted fence. The coding
skill picks worker labels on such a turn, for its first pick and for a
quota takeover.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Measured at this head on Linux/Python 3.12.13.

- `UV_NO_SYNC=1 uv run --no-sync pytest -q -n 12`: **33,767 passed, 178
skipped, 2 xfailed; 9 failed and 1 error.** Eight of the failures are
bound to this box and fail the same way on current main, measured the
same day: five provider tests that read the box's proxy environment, a
Node.js path case, a LibreOffice conversion, and a fixture that root can
read anyway. The ninth, with its teardown error, is the `remote_runs`
race tracked in #990; that test passed 5 of 5 runs alone, and its file
58 of 58.
- 13 new regression tests for Worker Table routing: 12 fail against the
previous head's source and all pass now; the thirteenth checks that a
Harness loaded mid-turn leaves the next turn's role plain, and passes on
both by design. The end-to-end case drives a real Persona-bound turn:
both enums offer `research-a` alone, dispatching `Raven-General` is
refused, and the role must name neither that agent nor the roster
fallback.
- Nine single-construct mutants, each applied to an isolated copy of the
tree, each fail at least one new test: the role ignoring the table, the
default path not reading the turn's table, the uncovered-work rule
removed, the generic fallback kept in the worker routing, the coding
skill reverted, and the mid-turn note never added, added on every call,
added without a dispatch path, or phrased apart from the role.
- The host identity on ordinary turns is byte-identical to the previous
head in all 16 rendered shapes: a launched sub-agent process or not,
with or without ownership lines, and every combination of the two
dispatch tools.
- Instance registry probe, run with this head and with main on one
synthetic file: reads leave the file byte-identical; one unrelated
`upsert_spawn` saves every former-name row as `Raven-General`, keeping
the more recently updated of two colliding spellings and the later one
in the file on a tie; main then finds none of those rows under `Raven`
or `raven`.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage.json> diff --base-ref origin/main --threshold 90`, over a
coverage run of 225 test files (8,952 passed): **99.40%**, 167/168
executable changed lines covered.
- `UV_NO_SYNC=1 make lint-types lint-imports check-large-files
check-source-language COMMIT_RANGE=origin/main...HEAD`: passed; all 10
import contracts kept.
- `UV_NO_SYNC=1 make check-commits`: both commit-message validators
passed; each also refused a dangling control commit with a malformed
subject.
- `ruff check` and `ruff format --check` over the branch's Python files
passed. `node ui-tui/scripts/gen-i18n.mjs --check` passed after the
rebase onto #1015, which also changed the i18n catalogue.
- `uv run --no-sync pre-commit run --from-ref origin/main --to-ref
HEAD`: every hook passed except the whole-TUI Prettier hook, which
reports only `newInstancePicker.tsx` and `subagentsHub.tsx`; both are
byte-identical to current main.
- Live-model replay and native Windows/macOS testing were not repeated
for these fixes.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- The host delegates work it previously performed itself, including
small scripts. Task-to-instance routing remains prompt-based; a task can
still stay with the host after it has already answered a follow-up
itself.
- In a conversation bound to a Persona or another Harness with workers,
work no worker's brief covers now waits for the user to add a worker or
change the conversation's Harness, rather than being carried out by the
host.
- Old agent spellings remain accepted. The instance registry's first
write after the upgrade, of any row for any session, saves every
former-name row in `subagent_instances.json` as `Raven-General`, and
keeps only the more recently updated row where two spellings of one
instance collide. Builds from before the rename cannot look those rows
up, so reverting the squash commit alone does not restore their instance
bindings: that needs the registry file as it stood before that first
write. Renaming rows back by hand cannot recover a colliding row the
write dropped. Transcripts, direct chats and stored graphs are not
rewritten.
- When multiple generic-agent configuration spellings coexist, only the
newest row contributes overrides. Fields supplied exclusively by an
older row are intentionally discarded.
- Moving the dispatch tools changes the tool-array cache prefix once
after upgrade.
- A plugin participant that removes `spawn` through `select_tools` can
still leave the graph tool pointing at an absent roster, because that
selection occurs after the registry's offer check. No bundled
participant does this.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-10-20 17:56:05 +08:00

236 lines
9.5 KiB
Python

"""Atomic agent-config write path."""
from __future__ import annotations
import json
from pathlib import Path
import pytest
from pydantic import ValidationError
from raven.config.agent_names import GENERIC_AGENT
from raven.config.update_subagents import (
add_agent,
get_agents,
remove_agent,
set_agents,
)
def _cfg(tmp_path: Path) -> Path:
return tmp_path / "config.json"
def _entries(path: Path) -> list[dict]:
return json.loads(path.read_text())["subagents"]["agents"]
def test_set_and_get_roundtrip(tmp_path: Path) -> None:
p = _cfg(tmp_path)
set_agents(
[
{"name": "claude_code", "kind": "cli", "command": "claude -p {prompt}"},
{"name": "mirothinker", "kind": "openai", "base_url": "http://x/v1", "model": "m"},
],
config_path=p,
)
got = get_agents(config_path=p)
assert [g["name"] for g in got] == ["claude_code", "mirothinker"]
# written under the current key, not the pre-rename one
raw = json.loads(p.read_text())
assert "agents" in raw["subagents"]
assert "thirdParty" not in raw["subagents"]
def test_invalid_kind_rejected_and_not_written(tmp_path: Path) -> None:
p = _cfg(tmp_path)
with pytest.raises(ValidationError):
set_agents([{"name": "bad", "kind": "nope"}], config_path=p)
assert not p.exists() # nothing written on validation failure
def test_duplicate_names_rejected(tmp_path: Path) -> None:
p = _cfg(tmp_path)
with pytest.raises(ValueError):
set_agents(
[
{"name": "dup", "kind": "cli", "command": "a {prompt}"},
{"name": "dup", "kind": "cli", "command": "b {prompt}"},
],
config_path=p,
)
def test_add_replaces_by_name(tmp_path: Path) -> None:
p = _cfg(tmp_path)
add_agent({"name": "codex", "kind": "cli", "command": "codex exec {prompt}"}, config_path=p)
add_agent({"name": "codex", "kind": "cli", "command": "codex exec2 {prompt}"}, config_path=p)
got = get_agents(config_path=p)
assert len(got) == 1 and got[0]["command"] == "codex exec2 {prompt}"
def test_remove(tmp_path: Path) -> None:
p = _cfg(tmp_path)
set_agents([{"name": "codex", "kind": "cli", "command": "codex {prompt}"}], config_path=p)
assert remove_agent("codex", config_path=p) is True
assert remove_agent("codex", config_path=p) is False
assert get_agents(config_path=p) == []
def test_preserves_other_config_sections(tmp_path: Path) -> None:
p = _cfg(tmp_path)
p.write_text(json.dumps({"providers": {"custom": {"apiBase": "http://x"}}, "agents": {}}))
set_agents([{"name": "codex", "kind": "cli", "command": "codex {prompt}"}], config_path=p)
raw = json.loads(p.read_text())
assert raw["providers"]["custom"]["apiBase"] == "http://x" # untouched
assert raw["subagents"]["agents"][0]["name"] == "codex"
def test_a_config_on_the_old_key_is_read_and_migrated_on_write(tmp_path: Path) -> None:
"""The rename must not orphan a stored config, and must not leave two lists.
Both spellings loading is what keeps an existing install working; writing only
the new one is what stops a later read having to guess which list is current.
"""
p = _cfg(tmp_path)
p.write_text(
json.dumps({"subagents": {"thirdParty": [{"name": "codex", "kind": "cli", "command": "codex {prompt}"}]}})
)
assert [g["name"] for g in get_agents(config_path=p)] == ["codex"]
add_agent({"name": "claude_code", "kind": "cli", "command": "claude -p {prompt}"}, config_path=p)
raw = json.loads(p.read_text())
assert "thirdParty" not in raw["subagents"]
assert [e["name"] for e in raw["subagents"]["agents"]] == ["codex", "claude_code"]
def test_a_builtin_name_cannot_be_claimed_by_another_transport(tmp_path: Path) -> None:
"""An override may retune a built-in agent; it may not replace it.
Redeclaring the name as cli leaves the in-process agent unreachable under a
name stored playbooks and instance records already point at.
"""
p = _cfg(tmp_path)
with pytest.raises(ValueError, match="built-in"):
set_agents([{"name": "raven", "kind": "cli", "command": "x {prompt}"}], config_path=p)
assert not p.exists()
# Overriding it as a built-in row is the supported edit.
set_agents([{"name": "raven", "kind": "builtin", "skills": ["local/web-search"]}], config_path=p)
assert get_agents(config_path=p)[0]["skills"] == ["local/web-search"]
class TestTheBuiltinNamesAreReserved:
"""Every spelling of the generic row's name, current or former, belongs to it,
and no transport may claim one -- but a config that already holds one must stay
editable.
`Raven` was accepted before the generic row was renamed to it, so real installs
can hold such an entry. Refusing every write while one is on disk would lock the
user out of the whole config surface, including the edit that removes it: the
guard rejects *introducing* a reserved name, not possessing one.
"""
@pytest.mark.parametrize("name", [GENERIC_AGENT, "Raven", "raven"])
@pytest.mark.parametrize("kind", ["cli", "openai"])
def test_a_new_entry_cannot_claim_either_spelling(self, tmp_path: Path, name: str, kind: str) -> None:
p = tmp_path / "config.json"
p.write_text(json.dumps({"subagents": {"agents": []}}))
with pytest.raises(ValueError, match="built-in"):
set_agents([{"name": name, "kind": kind, "command": "x {prompt}"}], config_path=p)
@pytest.mark.parametrize("name", [GENERIC_AGENT, "Raven", "raven"])
def test_an_acp_redeclaration_of_a_builtin_name_is_allowed(self, tmp_path: Path, name: str) -> None:
"""An acp row of a seed's name is the supported transport switch: the
generic agent is then served over this raven's own `raven acp`. An empty
command is kept as written -- merge_builtin_seeds fills the host command
at materialization, so a config file stays machine-portable."""
p = tmp_path / "config.json"
p.write_text(json.dumps({"subagents": {"agents": []}}))
set_agents([{"name": name, "kind": "acp", "command": ""}], config_path=p)
from raven.agent.subagent.builtin_agents import canonical_agent_name
stored = {canonical_agent_name(e["name"]): e for e in _entries(p)}
assert stored[GENERIC_AGENT]["kind"] == "acp"
def test_another_casing_is_not_reserved(self, tmp_path: Path) -> None:
# Only the spellings the row answers to are taken. `RAVEN` resolves to
# itself, collides with no seed, and is an ordinary name.
p = tmp_path / "config.json"
p.write_text(json.dumps({"subagents": {"agents": []}}))
set_agents([{"name": "RAVEN", "kind": "cli", "command": "x {prompt}"}], config_path=p)
assert [e["name"] for e in _entries(p)] == ["RAVEN"]
def test_an_entry_already_on_disk_does_not_block_an_unrelated_edit(self, tmp_path: Path) -> None:
p = tmp_path / "config.json"
p.write_text(
json.dumps(
{
"subagents": {
"agents": [
{"name": "Raven", "kind": "cli", "command": "x {prompt}", "description": "legacy"},
{"name": "Coder", "kind": "cli", "command": "x {prompt}", "description": "unrelated"},
]
}
}
)
)
# The edit the user is actually making is to Coder. Before this guard knew
# what was already on disk, the legacy row made every write fail -- including
# the one that would have removed it.
set_agents(
[
{"name": "Raven", "kind": "cli", "command": "x {prompt}", "description": "legacy"},
{"name": "Coder", "kind": "cli", "command": "x {prompt}", "description": "edited"},
],
config_path=p,
)
stored = {e["name"]: e for e in _entries(p)}
assert stored["Coder"]["description"] == "edited"
assert "Raven" in stored
def test_and_removing_the_legacy_entry_is_allowed(self, tmp_path: Path) -> None:
p = tmp_path / "config.json"
p.write_text(
json.dumps(
{
"subagents": {
"agents": [{"name": "Raven", "kind": "cli", "command": "x {prompt}", "description": "l"}]
}
}
)
)
set_agents([], config_path=p)
assert _entries(p) == []
def test_agents_on_machine_names_the_entries_that_run_there(tmp_path: Path) -> None:
from raven.config.update_subagents import agents_on_machine
p = _cfg(tmp_path)
set_agents(
[
{"name": "claude_code", "kind": "cli", "command": "claude -p {prompt}"},
{"name": "claude_code@box", "kind": "cli", "command": "claude -p {prompt}", "machine": "box"},
{"name": "codex@box", "kind": "cli", "command": "codex -p {prompt}", "machine": "box", "enabled": False},
{"name": "hermes@gpu", "kind": "cli", "command": "hermes -p {prompt}", "machine": "gpu"},
],
config_path=p,
)
assert agents_on_machine("box", config_path=p) == ["claude_code@box", "codex@box"]
assert agents_on_machine(" box ", config_path=p) == ["claude_code@box", "codex@box"]
# A disconnected entry dispatches nothing, so a removal is not refused over it.
assert agents_on_machine("box", only_enabled=True, config_path=p) == ["claude_code@box"]
assert agents_on_machine("nowhere", config_path=p) == []