## Summary Raven's host acts as the user's agent manager: it delegates production work, including small edits, to the agent whose ownership or description fits the task -- or, on a turn bound to a Worker Table, to the worker whose brief covers it -- keeps track of the task and its instance, routes follow-ups back to that instance, and gathers the results for the user. Its own tools support briefing, checking and coordination. - Add the resident `multi-agent-coding` skill for choosing coding agents, keeping one instance per task, passing results by file, isolating concurrent edits in worktrees, and handing over work when an agent exhausts its quota. - Keep the new host role authoritative in both the DAG skill's resident description and its full guide. A single task uses `spawn`, or a one-node graph when only the DAG dispatch path is available. - Rename the generic sub-agent to `Raven-General`, with `Raven` and `raven` retained as compatibility aliases. Instance records, handle locks, live activity, cancellation, preferences and RPC lookups use the same canonical identity. Historical spawn and DAG readers normalize names before liveness checks and serialization, so tasks and transcripts join the current instance roster. Concurrent alias calls preserve both turns instead of overwriting one conversation snapshot. The instance registry reads former-name rows under the current name and, on its next write, saves every loaded row under it; transcripts, direct chats and stored graphs stay as written. - Resolve built-in configuration aliases before merging fields: `Raven-General` wins over `Raven`, then `raven`, regardless of file order. Discard older rows wholesale with a warning, preventing stale model, permission or transport settings from leaking into the selected row. - Resume the most recently written message state and instance log across former names. Saved messages take precedence over an empty chat directory; equal timestamps prefer the current name and then the most recent former name. Existing files stay in place. Legacy spawn records, completed DAG manifests and unfinished graphs remain readable through the current name. - Emit the agent roster once at the end of `spawn`'s description. The registry places `run_subagent_dag` and then `spawn` last, including after late MCP or session tools. The graph tool renders the roster itself when `spawn` is unavailable. With a Worker Table bound, dispatch guidance selects worker labels from the live enum; the roster explains underlying capabilities. The tool descriptions and resident DAG guide follow the same rule. - On a turn bound to a Worker Table, such as a Persona session, `spawn` and the DAG nodes offer worker labels alone, and a generated Persona never includes the generic agent. The host role on such a turn routes production work to the worker whose brief covers it, by label, and puts work no worker covers to the user, who can add a worker or give the conversation another Harness, instead of doing it itself or naming an agent outside the enum. Ordinary turns render byte for byte as before. A Harness that `load_playbook` binds mid-turn takes effect after the identity was assembled, so that call's result carries the same routing as this system's trusted note, after the untrusted fence. The coding skill picks worker labels on such a turn, for its first pick and for a quota takeover. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Measured at this head on Linux/Python 3.12.13. - `UV_NO_SYNC=1 uv run --no-sync pytest -q -n 12`: **33,767 passed, 178 skipped, 2 xfailed; 9 failed and 1 error.** Eight of the failures are bound to this box and fail the same way on current main, measured the same day: five provider tests that read the box's proxy environment, a Node.js path case, a LibreOffice conversion, and a fixture that root can read anyway. The ninth, with its teardown error, is the `remote_runs` race tracked in #990; that test passed 5 of 5 runs alone, and its file 58 of 58. - 13 new regression tests for Worker Table routing: 12 fail against the previous head's source and all pass now; the thirteenth checks that a Harness loaded mid-turn leaves the next turn's role plain, and passes on both by design. The end-to-end case drives a real Persona-bound turn: both enums offer `research-a` alone, dispatching `Raven-General` is refused, and the role must name neither that agent nor the roster fallback. - Nine single-construct mutants, each applied to an isolated copy of the tree, each fail at least one new test: the role ignoring the table, the default path not reading the turn's table, the uncovered-work rule removed, the generic fallback kept in the worker routing, the coding skill reverted, and the mid-turn note never added, added on every call, added without a dispatch path, or phrased apart from the role. - The host identity on ordinary turns is byte-identical to the previous head in all 16 rendered shapes: a launched sub-agent process or not, with or without ownership lines, and every combination of the two dispatch tools. - Instance registry probe, run with this head and with main on one synthetic file: reads leave the file byte-identical; one unrelated `upsert_spawn` saves every former-name row as `Raven-General`, keeping the more recently updated of two colliding spellings and the later one in the file on a tie; main then finds none of those rows under `Raven` or `raven`. - `uv run --no-sync python scripts/coverage_gate.py --coverage-json <coverage.json> diff --base-ref origin/main --threshold 90`, over a coverage run of 225 test files (8,952 passed): **99.40%**, 167/168 executable changed lines covered. - `UV_NO_SYNC=1 make lint-types lint-imports check-large-files check-source-language COMMIT_RANGE=origin/main...HEAD`: passed; all 10 import contracts kept. - `UV_NO_SYNC=1 make check-commits`: both commit-message validators passed; each also refused a dangling control commit with a malformed subject. - `ruff check` and `ruff format --check` over the branch's Python files passed. `node ui-tui/scripts/gen-i18n.mjs --check` passed after the rebase onto #1015, which also changed the i18n catalogue. - `uv run --no-sync pre-commit run --from-ref origin/main --to-ref HEAD`: every hook passed except the whole-TUI Prettier hook, which reports only `newInstancePicker.tsx` and `subagentsHub.tsx`; both are byte-identical to current main. - Live-model replay and native Windows/macOS testing were not repeated for these fixes. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed ## Risk - The host delegates work it previously performed itself, including small scripts. Task-to-instance routing remains prompt-based; a task can still stay with the host after it has already answered a follow-up itself. - In a conversation bound to a Persona or another Harness with workers, work no worker's brief covers now waits for the user to add a worker or change the conversation's Harness, rather than being carried out by the host. - Old agent spellings remain accepted. The instance registry's first write after the upgrade, of any row for any session, saves every former-name row in `subagent_instances.json` as `Raven-General`, and keeps only the more recently updated row where two spellings of one instance collide. Builds from before the rename cannot look those rows up, so reverting the squash commit alone does not restore their instance bindings: that needs the registry file as it stood before that first write. Renaming rows back by hand cannot recover a colliding row the write dropped. Transcripts, direct chats and stored graphs are not rewritten. - When multiple generic-agent configuration spellings coexist, only the newest row contributes overrides. Fields supplied exclusively by an older row are intentionally discarded. - Moving the dispatch tools changes the tool-array cache prefix once after upgrade. - A plugin participant that removes `spawn` through `select_tools` can still leave the graph tool pointing at an absent roster, because that selection occurs after the registry's offer check. No bundled participant does this. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues N/A
215 lines
9.3 KiB
Python
215 lines
9.3 KiB
Python
"""A round's drills played at once: replicas start where the employee stands, on its revision, and are closed after."""
|
|
|
|
import asyncio
|
|
import json
|
|
import shutil
|
|
from types import SimpleNamespace
|
|
|
|
import pytest
|
|
|
|
from experimental.curator.harness import Artifact
|
|
from experimental.curator.raven_adapter.deployment import child_directory
|
|
from experimental.curator.raven_adapter.hosting.prepare import prepare_children
|
|
from experimental.curator.raven_adapter.strategy import Scopes
|
|
from experimental.curator.raven_adapter.worker import Execution
|
|
from experimental.iteration.protocols import Exchange
|
|
from experimental.simulation.agency import NAME as REVIEW
|
|
from experimental.simulation.agency import Agency
|
|
from experimental.simulation.employee import HOME, WORKDIR, Together, hire, replicate
|
|
from experimental.simulation.scenario import BUNDLED, Scenario
|
|
from raven.config.agent_names import GENERIC_AGENT
|
|
from raven.contracts.llm_provider import LLMResponse, ToolCallRequest
|
|
|
|
TRAVEL = BUNDLED / "travel_agency"
|
|
AUTHORED = Artifact(values={"memory.prompt": {"AGENTS.md": "Quote from the price list."}})
|
|
|
|
|
|
def config_file(tmp_path):
|
|
path = tmp_path / "config.json"
|
|
path.write_text(
|
|
json.dumps(
|
|
{
|
|
"providers": {"deepseek": {"api_key": "k"}},
|
|
"agents": {"defaults": {"model": "deepseek/x", "provider": "deepseek"}},
|
|
}
|
|
)
|
|
)
|
|
return path
|
|
|
|
|
|
def test_a_replica_copies_the_employee_as_it_stands_with_its_installed_revision(tmp_path):
|
|
scenario, config = Scenario.load(TRAVEL), config_file(tmp_path)
|
|
(tmp_path / "work").mkdir()
|
|
employee = hire(scenario, config, workdir=tmp_path / "work", root=tmp_path / "run")
|
|
employee.children = prepare_children(employee.baseline, employee.root, [GENERIC_AGENT])
|
|
employee.children[GENERIC_AGENT].artifact = Artifact(values={"memory.prompt": {"TOOLS.md": "Search first."}})
|
|
employee.artifact = AUTHORED
|
|
home = employee.baseline.config.workspace_path
|
|
(home / "skills" / "service-sop").mkdir(parents=True)
|
|
(home / "skills" / "service-sop" / "SKILL.md").write_text("SOP")
|
|
(home / "sessions").mkdir()
|
|
(home / "sessions" / "old.jsonl").write_text("{}")
|
|
(tmp_path / "work" / "handover").mkdir()
|
|
(tmp_path / "work" / "handover" / "ticket.md").write_text("earlier ticket")
|
|
saved = {"task_id": employee.baseline.task.id, "data": {}, "sessions": {"c1": {"stage": "S2"}}}
|
|
(employee.root / "planning.json").write_text(json.dumps(saved))
|
|
child_state = child_directory(employee.root, GENERIC_AGENT)
|
|
child_state.mkdir(parents=True, exist_ok=True)
|
|
(child_state / "action.json").write_text('{"held": 1}')
|
|
|
|
root = tmp_path / "run" / "replicas" / "1-student"
|
|
replica = replicate(employee, config, root)
|
|
assert replica.root == root.resolve() and replica.baseline.config.workspace_path == root / HOME
|
|
assert (root / HOME / "skills" / "service-sop" / "SKILL.md").read_text() == "SOP"
|
|
assert not (root / HOME / "sessions").exists()
|
|
assert (root / WORKDIR / "handover" / "ticket.md").read_text() == "earlier ticket"
|
|
assert replica.baseline.workdir == root / WORKDIR
|
|
assert (
|
|
replica.artifact == AUTHORED
|
|
and replica.children[GENERIC_AGENT].artifact == employee.children[GENERIC_AGENT].artifact
|
|
)
|
|
assert replica.children[GENERIC_AGENT].baseline.config.workspace_path == root / HOME / "subagents" / GENERIC_AGENT
|
|
# A replica under a new task id would refuse the employee's saved plan before any drill started.
|
|
assert replica.baseline.task == employee.baseline.task
|
|
assert Scopes("planning", replica.baseline.task, root / "planning.json", dict, per_session=True).saved["c1"] == {
|
|
"stage": "S2"
|
|
}
|
|
assert (child_directory(root, GENERIC_AGENT) / "action.json").read_text() == '{"held": 1}'
|
|
assert employee.artifact == AUTHORED and (tmp_path / "work" / "handover" / "ticket.md").is_file()
|
|
assert replica.revision_id == employee.revision_id
|
|
|
|
|
|
class Employee:
|
|
def __init__(self, workdir, artifact=AUTHORED, package="_curator_abc"):
|
|
self.baseline = SimpleNamespace(workdir=workdir, config=SimpleNamespace(workspace_path=workdir / "home"))
|
|
self.root = workdir
|
|
self.artifact, self.children, self.package = artifact, {}, package
|
|
self.started = self.closed = False
|
|
|
|
def records(self):
|
|
return [{"kind": "runtime.bound", "package": f"/runs/x/{self.package}"}] if self.started else []
|
|
|
|
async def start(self):
|
|
self.started = True
|
|
|
|
async def close(self):
|
|
self.closed = True
|
|
|
|
|
|
class Drill:
|
|
def __init__(self, name, gate=None, error=None):
|
|
self.name, self.gate, self.error, self.worker = name, gate, error, None
|
|
|
|
async def run(self, worker):
|
|
self.worker = worker
|
|
if self.gate:
|
|
await self.gate.wait()
|
|
if self.error:
|
|
raise self.error
|
|
return {self.name: []}
|
|
|
|
|
|
async def test_every_drill_runs_at_once_on_its_own_replica_and_the_employee_plays_none(tmp_path):
|
|
employee = Employee(tmp_path / "employee")
|
|
employee.started = True
|
|
made, placed = {}, {}
|
|
|
|
def spawn(worker, label):
|
|
assert worker is employee and not any(drill.worker for drill in drills.values())
|
|
made[label] = Employee(tmp_path / label)
|
|
return made[label]
|
|
|
|
gate = asyncio.Barrier(3)
|
|
drills = {name: Drill(name, gate) for name in ("family", "premium", "student")}
|
|
together = Together(drills, spawn, placed)
|
|
sessions = await asyncio.wait_for(together.run(employee), 5)
|
|
assert sessions == {"family": [], "premium": [], "student": []}
|
|
assert [drills[name].worker for name in drills] == [made[f"1-{name}"] for name in drills]
|
|
assert not employee.closed and all(replica.started and replica.closed for replica in made.values())
|
|
assert {name: where["workdir"] for name, where in placed.items()} == {
|
|
name: tmp_path / f"1-{name}" for name in drills
|
|
}
|
|
drills = {name: Drill(name) for name in drills}
|
|
together.trials = drills
|
|
await together.run(employee)
|
|
assert sorted(made) == [f"{round}-{name}" for round in (1, 2) for name in ("family", "premium", "student")]
|
|
|
|
|
|
async def test_a_replica_on_another_revision_never_plays_and_every_replica_is_closed(tmp_path):
|
|
employee = Employee(tmp_path / "employee")
|
|
employee.started = True
|
|
made = []
|
|
|
|
def spawn(worker, label):
|
|
made.append(Employee(tmp_path / label, package="_curator_old" if label.endswith("student") else "_curator_abc"))
|
|
return made[-1]
|
|
|
|
drills = {name: Drill(name) for name in ("family", "premium", "student")}
|
|
with pytest.raises(ValueError, match="bound _curator_old"):
|
|
await Together(drills, spawn).run(employee)
|
|
assert not any(drill.worker for drill in drills.values()) and all(replica.closed for replica in made)
|
|
|
|
|
|
async def test_a_failed_drill_stops_the_others_and_closes_the_replicas(tmp_path):
|
|
employee = Employee(tmp_path / "employee")
|
|
employee.started = True
|
|
made = []
|
|
|
|
def spawn(worker, label):
|
|
made.append(Employee(tmp_path / label))
|
|
return made[-1]
|
|
|
|
never = asyncio.Event()
|
|
drills = {"family": Drill("family", never), "student": Drill("student", error=RuntimeError("customer left"))}
|
|
with pytest.raises(RuntimeError, match="customer left"):
|
|
await asyncio.wait_for(Together(drills, spawn).run(employee), 5)
|
|
assert made and all(replica.closed for replica in made)
|
|
|
|
|
|
async def test_the_owner_reads_what_each_drill_filed_on_its_replica_and_nothing_it_found_there(tmp_path):
|
|
scenario = Scenario.load(TRAVEL)
|
|
employee = tmp_path / "employee"
|
|
(employee / "workdir" / "handover").mkdir(parents=True)
|
|
(employee / "workdir" / "handover" / "earlier.md").write_text("earlier ticket")
|
|
(employee / "home").mkdir()
|
|
placed = {}
|
|
for name, text in (("family", "family ticket"), ("student", "student ticket")):
|
|
replica = tmp_path / "replicas" / f"1-{name}"
|
|
shutil.copytree(employee, replica)
|
|
(replica / "workdir" / "handover" / "hold.md").write_text(text)
|
|
(replica / "home" / "notes").mkdir()
|
|
(replica / "home" / "notes" / "brief.md").write_text(f"{name} brief")
|
|
placed[name] = {"workdir": replica / "workdir", "home": replica / "home"}
|
|
|
|
def played(reply):
|
|
records = [{"kind": "runner.event", "event_type": "Text", "event": {"content": reply}}]
|
|
return [Exchange("Book it", Execution("t1", [], records, {}, "a"))]
|
|
|
|
rows = [{"id": criterion.id, "result": "pass"} for criterion in scenario.criteria]
|
|
verdict = LLMResponse(
|
|
content=None, tool_calls=[ToolCallRequest(REVIEW, REVIEW, {"verdicts": rows, "remark": "Fine."})]
|
|
)
|
|
requests = []
|
|
|
|
class Provider:
|
|
async def chat_with_retry(self, **kwargs):
|
|
requests.append(kwargs)
|
|
return verdict
|
|
|
|
agency = Agency(
|
|
scenario,
|
|
Provider(),
|
|
employee / "home" / "skills",
|
|
workdir=employee / "workdir",
|
|
plan="all",
|
|
uploads=employee / "home" / "uploads",
|
|
workdirs=placed,
|
|
)
|
|
agency.prepare()
|
|
await agency.evaluate({"family": played("Booked."), "student": played("Booked too."), "premium": played("Hi.")})
|
|
packet = json.loads(requests[0]["messages"][1]["content"])
|
|
assert packet["filed"] == {
|
|
name: {"workdir/handover/hold.md": f"{name} ticket", "home/notes/brief.md": f"{name} brief"}
|
|
for name in ("family", "student")
|
|
}
|