feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
"""raven.node.server: Raven's own file tools, answered over stdio.
|
|
|
|
|
|
|
|
|
|
The property everything else rests on is parity: a file call a node answers
|
|
|
|
|
comes back exactly as the same tool's answer on the host's own disk, footer
|
|
|
|
|
and limits included. Then the fence: a call reaches only the roots it names,
|
|
|
|
|
and one that names none is refused rather than read as "no fence".
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
from __future__ import annotations
|
|
|
|
|
|
|
|
|
|
import asyncio
|
|
|
|
|
import base64
|
|
|
|
|
import hashlib
|
|
|
|
|
import io
|
|
|
|
|
import json
|
|
|
|
|
import os
|
|
|
|
|
import signal
|
feat(*): lend the host's MCP servers to an acp agent on a machine through its node (#947)
## Summary
Adds MCP lending for remote ACP agents on top of the machine and
recovery
services already merged in #920, #922 and #934. The branch includes
current
main and its default-on registered-machine behavior.
What this does: an acp entry on a registered machine is handed the MCP
servers its row grants, the way an entry on this computer is, and the
servers themselves stay here with their definitions and keys.
- The agent on the machine runs the node's own bridge as a stdio MCP
server (`python -I -m raven.node --mcp --dir <node dir> --task <task>
--lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte
pumps around a connection to the daemon's socket, no MCP parsed, no
credential held, the standard library only.
- The daemon carries the bridge's frames as a *stream* to the host
holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6,
`raven/node/protocol.py`, `raven/node/server.py`). The daemon tells
which
task a bridge is by the process that connected -- the peer pid off the
socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the
parent chain to the agent the daemon started, which the bridge is under
and no other task's process is; the kernel supplies that pid and a peer
cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the
task it names must be the one the connection is a process of. Two tasks
on
one machine run as one user, so no name or file a bridge could present
would tell them apart; the connecting process does. One stream per
bridge, at most 16 per task; a frame
travels in pieces of at most 256 KiB characters, the last marked
(`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the
daemon relays and keeps nothing. A stream ends with either end, the
host's link or the task, and the other end is told.
- The host opens the endpoints for a machine row exactly as for a row
here
(`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in
the host's sandbox, HTTP auth added here), binds them to the machine,
dispatch and acquired agent task
(`NodePool.lend`/`unlend`) before handing out bridge stanzas. On
`mcp/open`
the host checks the daemon-authenticated task against that binding
before
connecting; it rechecks the grant and link after the socket opens. An
unlend, replacement grant or replaced link cannot authorize that pending
connection. The host then pumps the frames both ways
(`raven/node/client.py`).
`node/hello` names the bridge; a node serving without a daemon
(`--stdio`)
names none, and the dispatch then goes without MCP and says so.
- The reply closes with the servers lent and the machine's node, or with
why none were. Remote lending requires Unix-socket support on the host:
on Windows it is withheld before endpoint setup, the task continues, and
the reply explains why. The roster's "injectable MCP" matches that
support
(`session_mcp_delivered`). The cli lane on a machine still
withholds its servers (the next change); the openai lane has no tools.
Not in this change: the cli lane's MCP file on the machine; a stream
that
survives the link going (a bridge sees its server end, the agent reports
the call failed, and the next dispatch works again); MCP for a run
picked
up after this computer restarted (#934): it reports its calls fail.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Follow-up fix 05eaba67a, after the c9de53947 review:
- The nine tests the Windows review named now pin the host fact through
one
fixture, `unix_socket_host` (`tests/conftest.py`), which sets
`raven.mcp.bridge.unix_sockets_available` to true and re-points every
`raven.*` module that imported it by name; their bodies are unchanged.
- Found complete, not just the nine: the whole suite was run with
`asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted
before collection (a pytest plugin, controller and xdist workers), at
the
merge base `6b938a426` and at `c9de53947`, and the two failure sets
compared. 17 node ids failed at the head and not at the base: the nine,
and eight this change added that run the transport for real (the bridge
module, and two lending tests through the stand-in machine). Those eight
are guarded by host capability instead of pinned: the bridge module
skips
when `sys.platform == "win32" or not unix_sockets_available()`, the two
lending tests carry the existing `skipif(not unix_sockets_available())`.
After the fix the same run fails the same 66 node ids at the head as at
the base: nothing this change adds or touches fails on a host without
unix sockets.
- A bridge whose daemon side goes ends as streams end. The host can
close a
stream while a frame of the agent's is on its way; the daemon writes
`mcp/close` and hangs up, and the frame meets the closed socket, or the
read side is reset first. That exited 1 with `BrokenPipeError` or
`ConnectionResetError`; a connection-level end is now exit 0 with
nothing
said, a frame past the cap still exits 1 with its reason, both pumps are
awaited before the socket closes, and a daemon gone before the stream
opens is one sentence, not a traceback. Four deterministic cases with a
stand-in connection fail on `c9de53947` and pass here.
- The bridge test's stand-in agent outlives its bridge, as an agent
outlives
one of its MCP servers, so the later parts of
`test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins`
no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80
here.
- The focused set (node, MCP, remote, manager and roster files) -- 935
passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
simulation cases, which fail the same way on main on this computer);
diff coverage against main 97.67% (420/430). Real machine, node
reinstalled with this tree in a directory of the check's own: a stdlib
ACP
agent's bridge reached a host-held server and 1 MiB and 8 MiB results
crossed whole, bridge exit 0; nothing was left on the machine.
Follow-up fix c9de53947, after the same-user token finding:
- The daemon no longer uses a token. `agent/launch` mints and writes
none,
`mcp.token` and the `hmac` check are gone, and `Daemon.open_stream`
identifies the bridge by the connecting process instead (`_peer_pid` +
`Daemon.task_of`). The same-user path the reviewer kept open -- a peer
reading another task's `mcp.token` and opening its stream -- is closed,
because identity is the connection the kernel reports, not a readable
secret.
- Tests, each failing on the unfixed tree first: a bridge placed under
task
B that names task A is refused (`the connection is a process of task
'agent@box-2', not 'agent@box-1'`), as are a process under no task, an
unplaced connection, and a task the daemon does not run
(`tests/test_node_server.py`); a real-OS test launches an agent, takes
its
real child, and asserts the child resolves to the task while the test's
own
process resolves to none (runs on Linux in CI); `_peer_pid` is checked
against a real socketpair; the in-process and subprocess bridge tests
now
identify by the connecting process, and a bridge this test spawns
itself,
or another task's agent spawns, never opens the stream.
- Windows withholding now covers a row on this computer too, not only a
machine row: `session_mcp_delivered` drops the `machine and` guard and
answers no whenever `unix_sockets_available()` is false, so a local acp
row is no longer advertised as injectable and no longer raises at
dispatch
on a host without the transport; `_mcp_endpoints` yields nothing before
opening an endpoint and `_mcp_note` explains it. The two tests the
Windows
review named now guard on `unix_sockets_available()`, and the
withholding
test is parameterized over a machine row and a local row.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over the node/MCP/remote suite -- 97.63% changed-line coverage
(412/422).
`ruff`, `ty` on the changed modules, and the import-cycle budget pass.
- Real machine (ravenx2, this head, node reinstalled, digest
fdbe132e76f0):
a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB
and
an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the
connecting-process identity admits the legitimate bridge end to end on
real Linux; the cross-task refusal on Linux is the real-OS test above,
which CI runs there.
Earlier review fixes (4dc5b3e1f, the task-to-grant binding). Its
host-side
binding of each dispatch's endpoints to the acquired agent task is kept;
the
daemon-side token this run verified was since replaced by the
connecting-process
identity above (c9de53947), so the token-named tests and reproduction
below no
longer describe the current tree.
- `uv run --no-sync pytest tests/test_node_client.py
tests/test_node_server.py tests/test_node_bridge.py
tests/test_node_relay.py tests/test_node_bundle.py
tests/test_node_install.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py
tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict
--cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report=
--tb=short` -- 1495 passed and one minimal-node stdio response test
failed;
strict idle accounting also flagged the assembly test. Both cases pass
together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or
timing limits were weakened. The new authorization, revocation and host
compatibility tests all pass.
- The five focused cases failed on the unfixed implementation first: a
different authenticated task selecting another grant, a stream with no
task, each missing Unix socket API, and an astral Unicode chunk over the
node's byte limit. Regressions also cover revocation, grant replacement
and link replacement while an endpoint connection is opening, and a real
remote turn continuing when both host Unix socket APIs are absent.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 97.94% (381/389 changed executable lines); this fix alone is 100%
(40/40). Python lint, type, dependency
and import checks pass; all 10 import contracts hold. The scope checks
for commit messages, source language and file sizes pass. Both RPC
generators pass `--check` after integrating current main.
- A real Linux node with two concurrent tasks and a host loopback
endpoint:
before, the attacker's own task token plus the victim's public grant
names returns the victim endpoint's marker; after, it returns no data
and the endpoint is never reached. Its own grant works, unknown and
revoked grants fail, and the same attack remains denied after the relay
is killed and an ordinary ping reattaches both tasks. A full chunk of
262144 astral characters was lost/corrupted before and arrives intact
after. The final documented source tree was reinstalled and checked
again.
- Real ACP dispatch through the node's bridge reaches a host-held stdio
MCP tool and returns its marker; its grant is removed when the turn
ends.
With the host Unix socket APIs removed to simulate Windows, the remote
agent still completes, receives no MCP, explains the host limitation,
and the host MCP ledger records no extra call. This is not a native
Windows run. Dedicated test tasks, host endpoints and bridge processes
are cleaned up.
- CONTEXT.md, module/field docstrings, both collaboration guides, the
external design notes and this description were checked against the
implementation. Main's no-debug-switch behavior and full-turn recovery
documentation are retained.
Full combined-tree verification: `uv run --no-sync pytest -n 4
--tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure
reproduces on current main in a separate checkout: host CLI color
rendering,
LibreOffice detection and the two simulation cases. No other test fails.
The minimal-node response test that failed in the focused parallel
selection
passes in this full run. Diff coverage: 97.94% (381/389 executable
changed lines).
Earlier author verification:
- `uv run pytest tests/test_node_bridge.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_bundle.py tests/test_node_install.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py`
-- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge
and a stdio node naming none, the token written owner-only beside the
record, a stream opened with the task's token and refused with another
or for a task not running, refused with no host holding the task,
refused past 16 per task, one per bridge connection, the pieces of a
frame crossing both ways as they are and a stranger carrying nothing,
and every way a stream ends (the bridge closing, the host closing, the
bridge's connection ending, the host's link ending, the agent ending)
with the other end told; the bridge as a process against a real daemon
over its socket: a frame to the host and one back whole, a large frame
each way in pieces, the agent closing its stdin ending the stream and
the bridge leaving clean, the host closing the stream ending the bridge,
a wrong token and a missing daemon each refused on stderr with exit 1;
the stand-in machine with the real daemon and the real bridge: an acp
stub there that connects to its `mcpServers` and calls a tool reaches a
stdio server here and answers with its result, the stanza handed to it
naming the node's bridge, the task, the dispatch and the server and
nothing of the server's definition or socket, the ledger here showing
one call, no bridge outliving the dispatch; the reply's note and the
roster predicate for a machine row.
- `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed,
4 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`,
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`
and
`test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`,
which fail
the same way on main on this machine.
- Real machine (a registered Linux machine with Claude Code signed in,
the node reinstalled with the code before and after):
A28, before (the
code of #934's head and its node): the turn closed with "MCP servers
'probe' were not lent", Claude Code said no tool of that name was
connected, and the server here saw no call; after: `PROBE_OK:r2-lend`
in 9.4 s, the server here recorded the call, and the turn closed with
"lent to acp agent ... through the Raven node on machine ...". A34:
after
the turn the machine's process list shows no bridge, a grep of its node
directory finds nothing of the server's command or of this computer's
socket directory, and no endpoint socket is left here. A29: with the
tool switched off here, Claude Code answered UNAVAILABLE and the server
saw no call. A30: a 40 s tool call under way, the bridge visible in the
machine's process list, then cancelled: 3 s later no bridge there and
no socket here. A31: the relay ssh killed during a 20 s call: the turn
ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next
dispatch reached the server again in 7.6 s. A32: five calls in one turn
all reached the server and came back; an 8 MiB answer: the server here
produced it, and a plain ACP agent on the machine (a stdlib stub that
speaks the stanza's bridge directly) received it whole through the node
-- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude
Code itself closed the server on that answer and reported the call
failed: the agent's own cap on a tool result, not the link's.
- `ruff`, the commit-message and PR-body checks -- clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese collaboration guides describe lending,
task-bound authorization,
Windows withholding, and the existing CLI/restart limitations.
## Risk
Registered-machine support is enabled by configuration, without the
removed
trial environment switch. An ACP entry on a machine receives its granted
MCP servers on a Unix-socket-capable host; the host's normal MCP policy
still
applies. This intentionally exposes those tools to the selected remote
agent.
The server definition, credentials and endpoint paths remain on the
host.
The daemon identifies a bridge's task by the connecting process (the
peer
pid the kernel reports, walked to the agent it started), not by any
token or
name the bridge presents, so a peer task sharing the OS account cannot
open
another task's stream; the host then authorizes that task against the
dispatch/server grant. A system that reports no peer pid opens no
stream.
On Windows hosts remote lending is withheld explicitly, without aborting
the
remote task. Protocol 6 remains the MCP wire version; host and node must
also
have the same code digest, so updated nodes must be reinstalled.
Rollback is
a revert; the machine then carries no stream and receives no bridge
stanza.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#934 (merged); #920 and #922 (merged).
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-15 15:37:50 +08:00
|
|
|
import socket
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
import subprocess
|
|
|
|
|
import sys
|
|
|
|
|
import threading
|
|
|
|
|
import time
|
|
|
|
|
from pathlib import Path
|
|
|
|
|
from typing import Any
|
|
|
|
|
|
|
|
|
|
import pytest
|
|
|
|
|
|
|
|
|
|
if sys.platform == "win32":
|
|
|
|
|
pytest.skip("the daemon is POSIX: it locks with fcntl and listens on a unix socket", allow_module_level=True)
|
|
|
|
|
|
|
|
|
|
from raven.agent.tools.file_search import FindTool, GrepTool
|
|
|
|
|
from raven.agent.tools.filesystem import ListDirTool, ReadFileTool
|
|
|
|
|
from raven.node import __main__ as node_main
|
|
|
|
|
from raven.node import bundle, protocol, server
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture
|
|
|
|
|
def tree(tmp_path: Path) -> Path:
|
|
|
|
|
root = tmp_path / "work"
|
|
|
|
|
(root / "sub").mkdir(parents=True)
|
|
|
|
|
(root / "big.txt").write_text("".join(f"line {i} of the file\n" for i in range(1, 501)), encoding="utf-8")
|
|
|
|
|
(root / "sub" / "a.py").write_text("needle = 1\nhay = 2\n", encoding="utf-8")
|
|
|
|
|
(root / "b.md").write_text("hay\n", encoding="utf-8")
|
|
|
|
|
(tmp_path / "secret.txt").write_text("top secret\n", encoding="utf-8")
|
|
|
|
|
return root
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _local(cls: Any, root: Path) -> Any:
|
|
|
|
|
"""The tool as the host would run it on its own disk, fenced to ``root``."""
|
|
|
|
|
return cls(workspace=Path.home(), allowed_dirs=(root,), follow_binding=False)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def _ask(method: str, params: Any = None, *, request_id: Any = 1) -> dict[str, Any] | None:
|
|
|
|
|
frame: dict[str, Any] = {"jsonrpc": "2.0", "id": request_id, "method": method}
|
|
|
|
|
if params is not None:
|
|
|
|
|
frame["params"] = params
|
|
|
|
|
return await server.handle(frame)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def _result(method: str, params: dict[str, Any]) -> dict[str, Any]:
|
|
|
|
|
answer = await _ask(method, params)
|
|
|
|
|
assert answer is not None and "result" in answer, answer
|
|
|
|
|
return answer["result"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def _refusal(method: str, params: Any) -> tuple[int, str]:
|
|
|
|
|
answer = await _ask(method, params)
|
|
|
|
|
assert answer is not None and "error" in answer, answer
|
|
|
|
|
return answer["error"]["code"], answer["error"]["message"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- parity -------------------------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_read_through_the_node_is_the_read_the_host_would_have_made(tree):
|
|
|
|
|
path = str(tree / "big.txt")
|
|
|
|
|
|
|
|
|
|
got = await _ask(protocol.READ, {"path": path, "offset": 200, "limit": 5, "roots": [str(tree)]})
|
|
|
|
|
|
|
|
|
|
want = await _local(ReadFileTool, tree).execute(path, offset=200, limit=5)
|
|
|
|
|
digest = hashlib.sha256((tree / "big.txt").read_bytes()).hexdigest()
|
|
|
|
|
assert got == {"jsonrpc": "2.0", "id": 1, "result": {"text": want, "sha256": digest}}
|
|
|
|
|
assert "line 200 of the file" in want and "line 199 of the file" not in want
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_read_with_no_offset_starts_at_the_first_line(tree):
|
|
|
|
|
path = str(tree / "b.md")
|
|
|
|
|
|
|
|
|
|
got = await _result(protocol.READ, {"path": path, "roots": [str(tree)]})
|
|
|
|
|
|
|
|
|
|
assert got == {
|
|
|
|
|
"text": await _local(ReadFileTool, tree).execute(path),
|
|
|
|
|
"sha256": hashlib.sha256(b"hay\n").hexdigest(),
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_list_find_and_grep_answer_as_their_tools_do(tree):
|
|
|
|
|
roots = [str(tree)]
|
|
|
|
|
|
|
|
|
|
listed = await _result(protocol.LIST, {"path": str(tree), "recursive": True, "max_entries": 10, "roots": roots})
|
|
|
|
|
found = await _result(protocol.FIND, {"pattern": "*.py", "path": str(tree), "limit": 5, "roots": roots})
|
|
|
|
|
grepped = await _result(
|
|
|
|
|
protocol.GREP,
|
|
|
|
|
{
|
|
|
|
|
"pattern": "HAY",
|
|
|
|
|
"path": str(tree),
|
|
|
|
|
"glob": "*.py",
|
|
|
|
|
"output_mode": "content",
|
|
|
|
|
"case_insensitive": True,
|
|
|
|
|
"context": 1,
|
|
|
|
|
"limit": 5,
|
|
|
|
|
"roots": roots,
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
assert listed == {"text": await _local(ListDirTool, tree).execute(str(tree), recursive=True, max_entries=10)}
|
|
|
|
|
assert found == {"text": await _local(FindTool, tree).execute("*.py", path=str(tree), limit=5)}
|
|
|
|
|
assert grepped == {
|
|
|
|
|
"text": await _local(GrepTool, tree).execute(
|
|
|
|
|
"HAY", path=str(tree), glob="*.py", output_mode="content", case_insensitive=True, context=1, limit=5
|
|
|
|
|
)
|
|
|
|
|
}
|
|
|
|
|
assert "a.py" in found["text"] and "hay = 2" in grepped["text"] and "b.md" not in grepped["text"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_an_image_comes_back_with_its_blocks(tree):
|
|
|
|
|
from PIL import Image
|
|
|
|
|
|
|
|
|
|
picture = tree / "dot.png"
|
|
|
|
|
Image.new("RGB", (2, 2), "red").save(picture)
|
|
|
|
|
|
|
|
|
|
got = await _result(protocol.READ, {"path": str(picture), "roots": [str(tree)]})
|
|
|
|
|
|
|
|
|
|
want = await _local(ReadFileTool, tree).execute(str(picture))
|
|
|
|
|
assert got == {
|
|
|
|
|
"text": want.model_text,
|
|
|
|
|
"display": want.display_text,
|
|
|
|
|
"ok": True,
|
|
|
|
|
"blocks": want.blocks,
|
|
|
|
|
"sha256": hashlib.sha256(picture.read_bytes()).hexdigest(),
|
|
|
|
|
}
|
|
|
|
|
assert [block["type"] for block in got["blocks"]] == ["text", "image_url"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_relative_path_and_a_tilde_root_mean_the_home_directory(tmp_path, monkeypatch):
|
|
|
|
|
home = tmp_path / "home"
|
|
|
|
|
home.mkdir()
|
|
|
|
|
(home / "notes.txt").write_text("from home\n", encoding="utf-8")
|
|
|
|
|
monkeypatch.setenv("HOME", str(home))
|
|
|
|
|
|
|
|
|
|
got = await _result(protocol.READ, {"path": "notes.txt", "roots": ["~"]})
|
|
|
|
|
|
|
|
|
|
assert "from home" in got["text"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- the fence ----------------------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_file_outside_the_roots_is_not_read(tree):
|
|
|
|
|
outside = str(tree.parent / "secret.txt")
|
|
|
|
|
|
|
|
|
|
got = await _result(protocol.READ, {"path": outside, "roots": [str(tree)]})
|
|
|
|
|
|
|
|
|
|
assert "top secret" not in got["text"]
|
|
|
|
|
assert got == {"text": await _local(ReadFileTool, tree).execute(outside)}, "a refusal names no digest"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_link_inside_the_roots_does_not_lead_out_of_them(tree):
|
|
|
|
|
(tree / "link.txt").symlink_to(tree.parent / "secret.txt")
|
|
|
|
|
|
|
|
|
|
got = await _result(protocol.READ, {"path": str(tree / "link.txt"), "roots": [str(tree)]})
|
|
|
|
|
|
|
|
|
|
assert "top secret" not in got["text"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
|
|
|
("roots", "said"),
|
|
|
|
|
[
|
|
|
|
|
(None, "roots must be a non-empty list of directories"),
|
|
|
|
|
([], "roots must be a non-empty list of directories"),
|
|
|
|
|
("/", "roots must be a non-empty list of directories"),
|
|
|
|
|
([7], "roots must be a non-empty list of directories"),
|
|
|
|
|
([" "], "roots must be a non-empty list of directories"),
|
|
|
|
|
(["relative/dir"], "root 'relative/dir' is not an absolute directory"),
|
|
|
|
|
],
|
|
|
|
|
)
|
|
|
|
|
async def test_a_call_that_names_no_usable_roots_is_refused(tree, roots, said):
|
|
|
|
|
params: dict[str, Any] = {"path": str(tree / "b.md")}
|
|
|
|
|
if roots is not None:
|
|
|
|
|
params["roots"] = roots
|
|
|
|
|
|
|
|
|
|
assert await _refusal(protocol.READ, params) == (-32602, said)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- what a request may carry ---------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
|
|
|
("method", "params", "said"),
|
|
|
|
|
[
|
|
|
|
|
(protocol.READ, {"path": ""}, "path must be a non-empty string"),
|
|
|
|
|
(protocol.READ, {"path": 3}, "path must be a non-empty string"),
|
|
|
|
|
(protocol.READ, {"path": "b.md", "offset": True}, "offset must be a whole number"),
|
|
|
|
|
(protocol.READ, {"path": "b.md", "limit": "5"}, "limit must be a whole number"),
|
|
|
|
|
(protocol.LIST, {"path": ".", "recursive": "yes"}, "recursive must be true or false"),
|
|
|
|
|
(protocol.FIND, {"pattern": ""}, "pattern must be a non-empty string"),
|
|
|
|
|
(protocol.GREP, {"pattern": "x", "glob": 3}, "glob must be a string"),
|
|
|
|
|
(protocol.GREP, {"pattern": "x", "output_mode": 3}, "output_mode must be a string"),
|
|
|
|
|
(protocol.GREP, {"pattern": "x", "context": 1.5}, "context must be a whole number"),
|
|
|
|
|
],
|
|
|
|
|
)
|
|
|
|
|
async def test_a_malformed_field_is_refused_by_name(tree, method, params, said):
|
|
|
|
|
assert await _refusal(method, {**params, "roots": [str(tree)]}) == (-32602, said)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_params_that_are_not_an_object_are_refused():
|
|
|
|
|
assert await _refusal(protocol.READ, ["b.md"]) == (-32602, "params must be an object")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_an_unknown_method_is_refused():
|
|
|
|
|
assert await _refusal("fs/nothing", {}) == (-32601, "unknown method 'fs/nothing'")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_failure_inside_a_tool_is_an_answer_and_the_node_keeps_serving(monkeypatch):
|
|
|
|
|
async def broken(params: dict[str, Any]) -> dict[str, Any]:
|
|
|
|
|
raise RuntimeError("boom")
|
|
|
|
|
|
|
|
|
|
monkeypatch.setitem(server._HANDLERS, protocol.READ, broken)
|
|
|
|
|
|
|
|
|
|
assert await _refusal(protocol.READ, {}) == (-32603, "RuntimeError: boom")
|
|
|
|
|
assert (await _result(protocol.HELLO, {}))["protocol"] == protocol.PROTOCOL
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
|
|
|
"frame",
|
|
|
|
|
[
|
|
|
|
|
{"jsonrpc": "2.0", "method": protocol.HELLO},
|
|
|
|
|
{"jsonrpc": "2.0", "id": None, "method": protocol.HELLO},
|
|
|
|
|
{"jsonrpc": "2.0", "id": 1, "method": 5},
|
|
|
|
|
{"jsonrpc": "2.0", "id": 1, "result": {}},
|
|
|
|
|
[{"jsonrpc": "2.0", "id": 1, "method": protocol.HELLO}],
|
|
|
|
|
],
|
|
|
|
|
)
|
|
|
|
|
async def test_nothing_is_owed_for_a_notification_or_a_frame_that_is_not_a_request(frame):
|
|
|
|
|
assert await server.handle(frame) is None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- the handshake --------------------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_hello_says_what_the_host_checks_before_using_a_node(monkeypatch, tmp_path):
|
|
|
|
|
monkeypatch.setenv("HOME", str(tmp_path))
|
|
|
|
|
|
|
|
|
|
hello = await _result(protocol.HELLO, {"protocol": protocol.PROTOCOL})
|
|
|
|
|
|
|
|
|
|
assert hello["protocol"] == protocol.PROTOCOL
|
|
|
|
|
assert hello["digest"] == bundle.digest()
|
|
|
|
|
assert hello["home"] == str(tmp_path)
|
|
|
|
|
assert hello["pid"] == os.getpid()
|
|
|
|
|
assert hello["python"].count(".") == 2
|
|
|
|
|
assert isinstance(hello["rg"], bool)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_hello_from_a_host_of_another_protocol_is_refused():
|
|
|
|
|
assert await _refusal(protocol.HELLO, {"protocol": protocol.PROTOCOL + 1}) == (
|
|
|
|
|
-32600,
|
|
|
|
|
f"this node speaks protocol {protocol.PROTOCOL}, not {protocol.PROTOCOL + 1}",
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_hello_says_whether_grep_runs_rg_here(monkeypatch):
|
|
|
|
|
from raven.agent.tools import file_search
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(file_search, "_resolve_rg", lambda: None)
|
|
|
|
|
assert (await _result(protocol.HELLO, {}))["rg"] is False
|
|
|
|
|
monkeypatch.setattr(file_search, "_resolve_rg", lambda: "/opt/rg")
|
|
|
|
|
assert (await _result(protocol.HELLO, {}))["rg"] is True
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- the line loop --------------------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_serve_answers_line_by_line_and_skips_what_it_cannot_read():
|
|
|
|
|
reader = asyncio.StreamReader(limit=256)
|
|
|
|
|
for line in (
|
|
|
|
|
b"\n",
|
|
|
|
|
b"x" * 400 + b"\n",
|
|
|
|
|
b"not json\n",
|
|
|
|
|
json.dumps({"jsonrpc": "2.0", "method": protocol.HELLO}).encode() + b"\n",
|
|
|
|
|
json.dumps({"jsonrpc": "2.0", "id": 7, "method": protocol.HELLO}).encode() + b"\n",
|
|
|
|
|
json.dumps({"jsonrpc": "2.0", "id": 8, "method": "fs/nothing"}).encode(),
|
|
|
|
|
):
|
|
|
|
|
reader.feed_data(line)
|
|
|
|
|
reader.feed_eof()
|
|
|
|
|
out = io.BytesIO()
|
|
|
|
|
|
|
|
|
|
await server.serve(reader, out)
|
|
|
|
|
|
|
|
|
|
answers = [json.loads(line) for line in out.getvalue().splitlines()]
|
|
|
|
|
assert [a["id"] for a in answers] == [None, 7, 8]
|
|
|
|
|
assert answers[0]["error"] == {"code": -32700, "message": "not a JSON object"}
|
|
|
|
|
assert answers[1]["result"]["protocol"] == protocol.PROTOCOL
|
|
|
|
|
assert answers[2]["error"]["code"] == -32601
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
_NOISY_NODE = """
|
|
|
|
|
import sys
|
|
|
|
|
import time
|
|
|
|
|
from raven.node import server
|
|
|
|
|
|
|
|
|
|
async def noisy(params):
|
|
|
|
|
print("a library talking on stdout")
|
|
|
|
|
return {"text": "ok"}
|
|
|
|
|
|
|
|
|
|
server._HANDLERS["x/noisy"] = noisy
|
|
|
|
|
sys.exit(server.run_stdio())
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _node(argv: list[str], requests: list[dict[str, Any]], home: Path) -> subprocess.CompletedProcess[str]:
|
|
|
|
|
return subprocess.run(
|
|
|
|
|
[sys.executable, *argv],
|
|
|
|
|
input="".join(json.dumps(r) + "\n" for r in requests),
|
|
|
|
|
capture_output=True,
|
|
|
|
|
text=True,
|
|
|
|
|
timeout=60,
|
|
|
|
|
env={**os.environ, "HOME": str(home)},
|
|
|
|
|
cwd=home,
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
def test_a_node_process_answers_until_its_stdin_closes(tree):
|
|
|
|
|
done = _node(
|
|
|
|
|
["-m", "raven.node", "--stdio"],
|
|
|
|
|
[
|
|
|
|
|
{"jsonrpc": "2.0", "id": 1, "method": protocol.HELLO, "params": {"protocol": protocol.PROTOCOL}},
|
|
|
|
|
{"jsonrpc": "2.0", "id": 2, "method": protocol.READ, "params": {"path": "b.md", "roots": [str(tree)]}},
|
|
|
|
|
],
|
|
|
|
|
tree,
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
assert done.returncode == 0, done.stderr
|
|
|
|
|
first, second = (json.loads(line) for line in done.stdout.splitlines())
|
|
|
|
|
assert first["result"]["digest"] == bundle.digest()
|
|
|
|
|
assert "hay" in second["result"]["text"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# A node with nothing importable beyond the standard library, its own code and
|
|
|
|
|
# the packages it is installed with: what the next change to raven/agent/tools
|
|
|
|
|
# must not break, since every installed node runs it.
|
|
|
|
|
_BUDGETED_NODE = """
|
|
|
|
|
import sys
|
|
|
|
|
allowed = set(sys.stdlib_module_names) | set(sys.modules) | {"raven", *%r}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class _Budget:
|
|
|
|
|
def find_spec(self, name, path=None, target=None):
|
|
|
|
|
# The build's own sysconfig data is the standard library's, under a name of the platform's.
|
|
|
|
|
if name.partition(".")[0] in allowed or name.startswith("_sysconfigdata"):
|
|
|
|
|
return None
|
|
|
|
|
raise ImportError(f"{name} is not among a node's requirements")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
sys.meta_path.insert(0, _Budget())
|
|
|
|
|
from raven.node import server
|
|
|
|
|
sys.exit(server.run_stdio())
|
|
|
|
|
"""
|
|
|
|
|
_MODULE_OF = {"pillow": "PIL"}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
def test_a_node_answers_every_file_call_with_only_its_requirements_importable(tree):
|
|
|
|
|
importable = tuple(_MODULE_OF.get(name, name) for name in protocol.REQUIREMENTS)
|
|
|
|
|
calls = [
|
|
|
|
|
(protocol.READ, {"path": "b.md"}),
|
|
|
|
|
(protocol.LIST, {"path": "."}),
|
|
|
|
|
(protocol.FIND, {"pattern": "*.py", "path": "."}),
|
|
|
|
|
(protocol.GREP, {"pattern": "needle", "path": "."}),
|
|
|
|
|
(protocol.STAT, {"path": "b.md", "hash": True}),
|
|
|
|
|
(protocol.WRITE, {"path": "new.txt", "content": "x\n"}),
|
|
|
|
|
(protocol.EDIT, {"path": "new.txt", "old_text": "x", "new_text": "y"}),
|
|
|
|
|
(protocol.PULL, {"path": "b.md"}),
|
|
|
|
|
]
|
|
|
|
|
requests = [{"jsonrpc": "2.0", "id": 1, "method": protocol.HELLO, "params": {"protocol": protocol.PROTOCOL}}]
|
|
|
|
|
requests += [
|
|
|
|
|
{"jsonrpc": "2.0", "id": n, "method": method, "params": {**params, "roots": [str(tree)]}}
|
|
|
|
|
for n, (method, params) in enumerate(calls, start=2)
|
|
|
|
|
]
|
|
|
|
|
|
|
|
|
|
done = _node(["-c", _BUDGETED_NODE % (importable,)], requests, tree)
|
|
|
|
|
|
|
|
|
|
assert done.returncode == 0, done.stderr
|
|
|
|
|
answers = [json.loads(line) for line in done.stdout.splitlines()]
|
|
|
|
|
assert [answer.get("error") for answer in answers] == [None] * len(requests), answers
|
|
|
|
|
assert answers[0]["result"]["digest"] == bundle.digest()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
def test_stdout_carries_only_frames_whatever_else_prints(tree):
|
|
|
|
|
done = _node(["-c", _NOISY_NODE], [{"jsonrpc": "2.0", "id": 1, "method": "x/noisy"}], tree)
|
|
|
|
|
|
|
|
|
|
assert done.returncode == 0, done.stderr
|
|
|
|
|
assert [json.loads(line) for line in done.stdout.splitlines()] == [
|
|
|
|
|
{"jsonrpc": "2.0", "id": 1, "result": {"text": "ok"}}
|
|
|
|
|
]
|
|
|
|
|
assert "a library talking on stdout" in done.stderr
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_run_stdio_serves_this_process_s_own_stdin_and_moves_print_to_stderr(monkeypatch):
|
|
|
|
|
read_end, write_end = os.pipe()
|
|
|
|
|
os.write(write_end, (json.dumps({"jsonrpc": "2.0", "id": 3, "method": protocol.HELLO}) + "\n").encode())
|
|
|
|
|
os.close(write_end)
|
|
|
|
|
frames = io.BytesIO()
|
|
|
|
|
|
|
|
|
|
class _Stdout:
|
|
|
|
|
buffer = frames
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(sys, "stdin", os.fdopen(read_end, "rb", buffering=0))
|
|
|
|
|
monkeypatch.setattr(sys, "stdout", _Stdout())
|
|
|
|
|
|
|
|
|
|
assert server.run_stdio() == 0
|
|
|
|
|
|
|
|
|
|
(answer,) = [json.loads(line) for line in frames.getvalue().splitlines()]
|
|
|
|
|
assert answer["id"] == 3 and answer["result"]["protocol"] == protocol.PROTOCOL
|
|
|
|
|
assert sys.stdout is sys.stderr
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- python -m raven.node ---------------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_version_prints_the_protocol_and_the_digest(capsys):
|
|
|
|
|
assert node_main.main(["--version"]) == 0
|
|
|
|
|
assert json.loads(capsys.readouterr().out) == {"protocol": protocol.PROTOCOL, "digest": bundle.digest()}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_stdio_serves(monkeypatch):
|
|
|
|
|
monkeypatch.setattr(server, "run_stdio", lambda: 0)
|
|
|
|
|
|
|
|
|
|
assert node_main.main(["--stdio"]) == 0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_a_daemon_refused_its_socket_s_place_says_so_in_one_line_and_leaves(tmp_path, monkeypatch, capsys):
|
|
|
|
|
def refuse(root: Path, *, idle_s: float) -> int:
|
|
|
|
|
raise PermissionError(f"{root} is in the way of the daemon")
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr("raven.node.server.run_daemon", refuse)
|
|
|
|
|
|
|
|
|
|
assert node_main.main(["--serve", "--dir", str(tmp_path)]) == 1
|
|
|
|
|
|
|
|
|
|
assert f"raven node: {tmp_path} is in the way of the daemon" in capsys.readouterr().err
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize("argv", [[], ["--stdio", "--version"]])
|
|
|
|
|
def test_a_node_is_started_for_exactly_one_purpose(argv, capsys):
|
|
|
|
|
with pytest.raises(SystemExit) as caught:
|
|
|
|
|
node_main.main(argv)
|
|
|
|
|
|
|
|
|
|
assert caught.value.code == 2
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- the link's own methods --------------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_ping_comes_back_with_what_it_carried():
|
|
|
|
|
assert await _result(protocol.PING, {"t": 12.5}) == {"t": 12.5}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_the_session_directory_is_made_and_resolved_here(tmp_path, monkeypatch):
|
|
|
|
|
monkeypatch.setenv("HOME", str(tmp_path))
|
|
|
|
|
|
|
|
|
|
got = await _result(protocol.SESSION_DIR, {"path": "~/raven-work/a b"})
|
|
|
|
|
|
|
|
|
|
assert got["path"] == str((tmp_path / "raven-work" / "a b").resolve())
|
|
|
|
|
assert Path(got["path"]).is_dir()
|
|
|
|
|
assert abs(got["since"] - time.time()) < 60, "this machine's clock, as the directory was made"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_session_directory_that_cannot_be_made_is_refused_with_the_reason(tmp_path):
|
|
|
|
|
(tmp_path / "f").write_text("a file\n", encoding="utf-8")
|
|
|
|
|
|
|
|
|
|
code, said = await _refusal(protocol.SESSION_DIR, {"path": str(tmp_path / "f" / "t1")})
|
|
|
|
|
|
|
|
|
|
assert code == -32603 and said.startswith(f"could not make {tmp_path / 'f' / 't1'}: ")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_the_machine_probe_reports_room_python_shell_and_load():
|
|
|
|
|
got = await _result(protocol.PROBE_MACHINE, {})
|
|
|
|
|
|
|
|
|
|
assert set(got) == {"disk_free_kb", "python", "shell", "uv", "load"}
|
|
|
|
|
assert str(Path.home()) in got["disk_free_kb"] and got["python"].count(".") == 2
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_node_drives_the_login_shells_the_host_does():
|
|
|
|
|
from raven.agent.subagent.backends.env import _DRIVABLE_SHELLS
|
|
|
|
|
|
|
|
|
|
assert server._DRIVABLE_SHELLS == _DRIVABLE_SHELLS
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_login_environment_is_captured_from_a_drivable_shell_and_left_alone_otherwise(tmp_path, monkeypatch):
|
|
|
|
|
monkeypatch.setenv("SHELL", "/bin/csh")
|
|
|
|
|
assert server.capture_login_env() == dict(os.environ), "a shell it cannot drive leaves ssh's environment"
|
|
|
|
|
|
|
|
|
|
bash = tmp_path / "bash"
|
|
|
|
|
bash.write_text("#!/bin/sh\nprintf 'PATH=/from/profile\\0X=1\\0garbage\\0'\n", encoding="utf-8")
|
|
|
|
|
bash.chmod(0o755)
|
|
|
|
|
monkeypatch.setenv("SHELL", str(bash))
|
|
|
|
|
assert server.capture_login_env() == {"PATH": "/from/profile", "X": "1"}
|
|
|
|
|
|
|
|
|
|
bash.write_text("#!/bin/sh\nprintf 'X=1\\0'\n", encoding="utf-8")
|
|
|
|
|
assert server.capture_login_env() == dict(os.environ), "a capture without a PATH is no capture"
|
|
|
|
|
bash.write_text("#!/bin/sh\nexit 1\n", encoding="utf-8")
|
|
|
|
|
assert server.capture_login_env() == dict(os.environ), "a failed capture leaves ssh's environment"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- agents the node starts -----------------------------------------------------------
|
|
|
|
|
|
|
|
|
|
# An agent for these tests: says one thing on stderr, answers every request
|
|
|
|
|
# with its params (or its environment, or a child it started), and leaves when
|
|
|
|
|
# its stdin closes.
|
|
|
|
|
_AGENT = """
|
|
|
|
|
import json, os, subprocess, sys
|
|
|
|
|
sys.stderr.write("agent: started\\n")
|
|
|
|
|
sys.stderr.flush()
|
|
|
|
|
for line in sys.stdin:
|
|
|
|
|
line = line.strip()
|
|
|
|
|
if not line:
|
|
|
|
|
continue
|
|
|
|
|
request = json.loads(line)
|
|
|
|
|
method = request.get("method")
|
|
|
|
|
if method == "env":
|
|
|
|
|
result = {"login": os.environ.get("FROM_LOGIN"), "entry": os.environ.get("FROM_ENTRY"), "cwd": os.getcwd()}
|
|
|
|
|
elif method == "spawn":
|
|
|
|
|
child = subprocess.Popen([sys.executable, "-c", "import time; time.sleep(120)"])
|
|
|
|
|
result = {"pid": child.pid}
|
|
|
|
|
elif method == "detach":
|
|
|
|
|
# In a session of its own, the way Claude Code's shell tool starts a command.
|
|
|
|
|
child = subprocess.Popen([sys.executable, "-c", "import time; time.sleep(120)"], start_new_session=True)
|
|
|
|
|
result = {"pid": child.pid}
|
|
|
|
|
else:
|
|
|
|
|
result = {"echo": request.get("params")}
|
|
|
|
|
print(json.dumps({"jsonrpc": "2.0", "id": request["id"], "result": result}), flush=True)
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class _Host:
|
|
|
|
|
"""The host's end of the connection: every line the node wrote, decoded."""
|
|
|
|
|
|
|
|
|
|
def __init__(self) -> None:
|
|
|
|
|
self.out = io.BytesIO()
|
|
|
|
|
self.node = server.Node(self.out)
|
|
|
|
|
|
|
|
|
|
def frames(self) -> list[dict[str, Any]]:
|
|
|
|
|
return [json.loads(line) for line in self.out.getvalue().splitlines() if line.strip()]
|
|
|
|
|
|
|
|
|
|
def notified(self, method: str) -> list[dict[str, Any]]:
|
|
|
|
|
return [frame["params"] for frame in self.frames() if frame.get("method") == method]
|
|
|
|
|
|
|
|
|
|
async def ask(self, method: str, params: dict[str, Any], request_id: int = 1) -> dict[str, Any]:
|
|
|
|
|
answer = await server.handle(
|
|
|
|
|
{"jsonrpc": "2.0", "id": request_id, "method": method, "params": params}, self.node
|
|
|
|
|
)
|
|
|
|
|
assert answer is not None
|
|
|
|
|
return answer
|
|
|
|
|
|
|
|
|
|
async def tell(self, method: str, params: dict[str, Any]) -> None:
|
|
|
|
|
assert await server.handle({"jsonrpc": "2.0", "method": method, "params": params}, self.node) is None
|
|
|
|
|
|
|
|
|
|
async def until(self, check: Any, *, timeout: float = 10.0) -> None:
|
|
|
|
|
deadline = time.monotonic() + timeout
|
|
|
|
|
while not check():
|
|
|
|
|
assert time.monotonic() < deadline, "did not happen in time"
|
|
|
|
|
await asyncio.sleep(0.02)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _gone(pid: int) -> bool:
|
|
|
|
|
try:
|
|
|
|
|
os.kill(pid, 0)
|
|
|
|
|
except ProcessLookupError:
|
|
|
|
|
return True
|
|
|
|
|
try:
|
|
|
|
|
done, _ = os.waitpid(pid, os.WNOHANG)
|
|
|
|
|
return done == pid
|
|
|
|
|
except ChildProcessError:
|
|
|
|
|
return False
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture
|
|
|
|
|
def agent(tmp_path: Path) -> str:
|
|
|
|
|
script = tmp_path / "agent.py"
|
|
|
|
|
script.write_text(_AGENT, encoding="utf-8")
|
|
|
|
|
return f"{sys.executable} {script}"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _frame(method: str, request_id: int = 1) -> str:
|
|
|
|
|
return json.dumps({"jsonrpc": "2.0", "id": request_id, "method": method})
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_an_agent_starts_in_its_root_in_the_login_environment_plus_the_entry_s_and_is_forwarded(
|
|
|
|
|
tmp_path, agent, monkeypatch
|
|
|
|
|
):
|
|
|
|
|
monkeypatch.setattr(server, "capture_login_env", lambda: {**os.environ, "FROM_LOGIN": "yes"})
|
|
|
|
|
host = _Host()
|
|
|
|
|
root = tmp_path / "root" / "deep"
|
|
|
|
|
|
|
|
|
|
started = await host.ask(
|
|
|
|
|
protocol.AGENT_LAUNCH, {"channel": 1, "command": agent, "cwd": str(root), "env": {"FROM_ENTRY": "1"}}
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
assert isinstance(started["result"]["pid"], int) and root.is_dir(), "the root is made for it"
|
|
|
|
|
await host.tell(protocol.AGENT_FRAME, {"channel": 1, "line": _frame("env", 5)})
|
|
|
|
|
await host.until(lambda: host.notified(protocol.AGENT_FRAME))
|
|
|
|
|
answer = json.loads(host.notified(protocol.AGENT_FRAME)[0]["line"])
|
|
|
|
|
assert answer["id"] == 5
|
|
|
|
|
assert answer["result"]["login"] == "yes" and answer["result"]["entry"] == "1"
|
|
|
|
|
assert Path(answer["result"]["cwd"]).resolve() == root.resolve()
|
|
|
|
|
await host.until(lambda: host.notified(protocol.AGENT_STDERR))
|
fix(node): an attach replays the agent's stderr past the line it last read (#977)
## Summary
Follow-up to LivXue's review note on #957 (the pick-up's binding at
`cli_agent.py:1393`). The machine's daemon kept an agent's stderr in the
task's record (`stderr.log`) but sent it only to the connection holding
the
task when a line was written, and `agent/attach` replayed `frames.jsonl`
alone. So what an agent said only on stderr while no connection held it
never
reached the host:
- **The session id a CLI names only on stderr** (hermes does). The
pick-up
#957 added binds a create's handle to the id in its output; for such a
CLI
it found none, warned "not resumable", and the next turn to the instance
opened a new session.
- **The reason a command failed.** A run picked up after this computer
stopped was reported as `exited 3` where the dispatch would have said
`exited 3: token expired`. This part predates #957.
- **The stderr written while a link was down**, on a run still being
watched, the same way.
The fix is in the node and the host's link to it (`raven/node/`):
- The daemon numbers stderr lines as it numbers frames (`agent/stderr`
carries `n`), and `agent/attach` takes an optional `stderr_after`: the
stderr lines past that one are replayed after the frames, under the lock
that forwards a new line, so none is sent twice or missed at the
handover.
It works the same for a live task, a finished one, and a record a daemon
that never ran the task reads. An attach that does not ask hears no
stderr,
as before; a `stderr_after` that is not a whole number is refused like a
bad `after`.
- The host's `ChannelEndpoint` counts the stderr lines it got
(`stderr_received`). A cli pick-up (`NodePool.attach`, a process that
read
none of them) asks from the first line; an acp pick-up from the line the
run's record kept as the prompt went out (`RemoteRun.stderr_received`,
follow-up commit below); a re-attach after a lost link
(`NodePool._reattach`) asks from the last one it read.
The cli dispatch and pick-up read stderr off the endpoint as before and
now
get all of it, so the reason and the session id are there. An acp
pick-up's
connection drains the same endpoint into the stderr tail it quotes in
its
errors and into its journal, so what the agent wrote during the turn
while
this computer was away reaches them too; the acp run record keeps the
stderr
line count as the prompt goes out, so the earlier prompts' lines are not
read again (follow-up commit below). No protocol bump:
the new fields are optional, and a node runs exactly the host's code
anyway
(protocol and digest are checked at every connection).
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `tests/test_node_stderr_replay.py`, new, 22 tests: the daemon numbers
and
keeps stderr lines; an attach replays exactly the lines past
`stderr_after`
(from a live task, a finished one, and a record a daemon that never ran
it
reads), none when not asked, none from a record with no `stderr.log`,
and
refuses a bad `stderr_after`; the endpoint counts by the node's numbers;
a
re-attach and a pick-up ask from the right line. End to end on the
stand-in machine (real daemon, stand-in ssh): a left cli run's reason on
stderr reaches its pick-up (said before and after this computer
stopped),
a left create that names its session only on stderr is bound to it by
the
pick-up (both orders), and a reason written while the link was down
reaches the run watching it. 18 fail on main (2c2718122); the 4 that
pass
there pin what does not change (no `stderr_after`, past the last line,
past the end, no `stderr.log`).
- Changed assertions: the four attach-parameter assertions in
`tests/test_node_client.py` and the forwarded-stderr one in
`tests/test_node_server.py` include the new fields.
- Full suite: 32065 passed, 173 skipped, 2 xfailed, 3 failed -- the same
3
that fail on main (2c2718122) on this computer when run alone there: a
host
without LibreOffice, and two simulation cases.
- Diff coverage: 100.00% (30/30 executable changed lines; `coverage
json`, then
`scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`).
- `ruff check`, `ruff format --check`, `ty check`, `lint-imports`,
pre-commit on the diff; `scripts/check_commit_messages.py`,
`scripts/check_source_language.py` and `scripts/check_large_files.py` on
`origin/main..HEAD`: clean.
- Docs: CONTEXT.md's Raven node entry, the protocol module's notes, and
the
machines guide's "Disconnects and restarts" section in English and
Chinese
now say the daemon keeps and replays stderr; the changelog has an entry
under Fixed. `tests/test_docs_site.py` and `tests/test_living_docs.py`
pass
(18), `mkdocs build --strict` passes.
- Real machine, ravenx2, an isolated node directory per tree. The host
process that dispatched was killed with SIGKILL and the run picked up by
a
new process; the link was dropped by killing the local ssh. The command
writes `token expired` (or `session slow-A1`) on stderr:
| Probe | main (2c2718122) | this branch |
|---|---|---|
| reason written after the host process was killed | `exited 3` |
`exited 3: token expired` |
| reason written before it was killed | `exited 3` | `exited 3: token
expired` |
| reason written while the link was down | `exited 3` | `exited 3: token
expired` |
| create left before its commit, session id only on stderr | "not
resumable", nothing bound | bound to `slow-A1`, no warning |
The machine was cleaned after: the probe's two daemons ended by pid, its
directory removed.
- The runs above are on main 2c2718122 (#957 merged). The branch was
then
rebased onto 400b30dfa, whose two new commits touch only TUI copy and
i18n; the node, docs and range checks were run again there (238 passed,
1
skipped). It merges cleanly with the open #964 and #965 (`git
merge-tree`).
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
- Follow-up commit e0468fa88, after the agent review's F1 (an acp
pick-up asked
for the stderr from the agent's first line, across every earlier prompt
of
its process): `RemoteRun.stderr_received`, taken in
`Tracker.prompt_sent`
off the endpoint's count, and passed through `NodePool.attach(...,
stderr_after=...)` by the pick-up; a cli pick-up keeps asking from 0; a
record from before the field reads with 0. Regression on the stand-in
machine
(`test_a_pick_up_reads_the_turn_s_own_stderr_and_not_what_the_prompts_before_it_wrote`):
two turns of 150 stderr lines, a third interrupted after its prompt, the
next start's pick-up -- before the change its tail held 200 lines, 50 of
them the second turn's; after it, the third turn's 150 and nothing else.
Unit tests pin the record, the pick-up's attach arguments and the
default
of 0, and the tracker's wiring. The related files 259 passed; full suite
32067 passed with the three failures this Mac always has, plus one test
whose stand-in `Tracker` lacked the new keyword, fixed in the same
commit.
Diff coverage of the changed lines: see CI.
## Risk
The replayed lines are the agent's own stderr, already kept in the
task's
owner-only record and already sent live to whichever connection held the
task; nothing new leaves the machine. `stderr_after` and `n` are
optional: an
attach without `stderr_after` behaves as before, and a line without a
number
counts on from the last. A cli pick-up replays the whole `stderr.log`,
as it
already replays every frame of the command; an acp pick-up replays the
lines
past the count its record kept as the prompt went out, so the replay is
bounded to the turn, and the connection keeps a bounded tail of it.
Rollback
is reverting the commit.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follow-up to #957, for this review note:
https://github.com/EverMind-AI/Raven/pull/957#discussion_r4282448689
---------
Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-17 21:08:32 +08:00
|
|
|
assert host.notified(protocol.AGENT_STDERR) == [{"channel": 1, "text": "agent: started", "n": 1}]
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
|
|
|
|
|
closed = await host.ask(protocol.AGENT_CLOSE, {"channel": 1}, request_id=2)
|
|
|
|
|
|
|
|
|
|
assert closed["result"] == {"rc": 0}, "stdin closed, the agent left on its own"
|
|
|
|
|
await host.until(lambda: host.notified(protocol.AGENT_EXIT))
|
|
|
|
|
assert host.notified(protocol.AGENT_EXIT) == [{"channel": 1, "rc": 0}]
|
|
|
|
|
assert host.node.agents == {}
|
|
|
|
|
|
|
|
|
|
|
feat(*): read, write and deliver files on a registered machine by address (#922)
## Summary
Third of four changes that let a Raven dispatch work to the machines in
its machine registry. #920 is merged; this branch is opened against main
ahead of #921, so the diff here includes #921's commits as well: review
from "read, write and deliver files on a registered machine by address"
on (the commits before it are #921's, carried here until it merges).
What this does:
- The file tools take a path on a registered machine as it is written
for scp -- `user@host:/path`, or `scp://user@host:port//path` when the
port is not 22 -- and answer it through the machine's daemon
(`raven/agent/tools/remote_files.py`, hooked from `filesystem.py` and
`file_search.py`): `read_file`, `list_dir`, `find` and `grep` run
Raven's own tools there and bring back the answer, so a read of fifty
lines of a large file moves those fifty lines; `write_file` and
`edit_file` write there, with the digest of that agent's own last read
-- the conversation's, or its sub-agent run's -- as the expected
state, so a file changed since it read it is refused and nothing is
written, and another agent's read does not stand in for its own. The
path is resolved against the registry
only, never this computer's ssh config, so one path names one machine
on any computer; one that matches no row is refused, not guessed.
- What is shown back as an address is only what the tool says about
paths -- its errors, its headers and footers, a write's account, a
diff's two header lines; a read's numbered lines, a search's matches
and context, and a diff's body come back as they are, so the model
edits against the bytes the file holds. The shell's refusal of typed
`ssh` to a registered machine (`raven/agent/tools/shell.py`,
`machine_exec.py`) reads the program in any spelling -- a path, a
backslash, and on a Windows host `ssh.exe`, `SSH.EXE` or a whole
`C:\...\ssh.exe`, with the backslashes the shell there keeps and
without the quotes it takes off (a quoted piece and a bare one next to
it making one word), so a quoted destination or port is matched like a
bare one -- and, with the lane on, names the file tools. `scp`, `sftp`
and `rsync` to a registered machine stay what they are on main: a copy
starts nothing there and passes exec's own question like any command,
and the file tools cannot carry a directory, a binary file or a large
one, so the copy stays the route for those. A drive-relative or
extended Windows path (`C:file`, `\\?\C:\...`, UNC) is this
computer's, not a machine nobody registered.
- Permissions (`raven/permissions/`): a path inside the directories
Raven's agents work in on that machine -- its session directory, its
`remoteCwd`, the row's listed `paths` -- is read without asking;
elsewhere a read asks once per machine and directory for the session.
A write there asks wherever it falls, as a write here does, and its
grant is for the one file, apart from any read. The daemon checks the
roots again on the machine, links included.
- `deliver_files` fetches one file by its address, whole and checked,
under a 20 MB cap, into `<workspace>/pulled/<machine>/<key>/<name>`,
the key a digest of the file's path on the machine, so two paths never
share a landing: not two files of one name from two directories, and
not two names this computer's filesystem reads as one (`Report.txt`
and `report.txt`, a trailing dot). The name beside the key is written
as one this computer keeps (a backslash, what Windows reserves in a
name, control characters and the percent sign itself as
percent-codes), and a landing that would not resolve under the
machine's delivery directory is refused before anything is made; a
bigger file is named with its size and read there instead. Only a file
under the directories this Raven's agents work in
on that machine is fetched: the tool asks nothing, and a read some
conversation was allowed is no evidence for this one, so a file
elsewhere is refused and read there instead. Nothing reclaims
`pulled/`: a delivery is opened from its landing for as long as its
record lasts, and a bound by size or age is left out of this change on
purpose.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_remote_file_tools.py
tests/test_permissions_remote_read.py tests/test_deliver_files_tool.py
tests/test_shell_machine_channel.py tests/test_ops_remote_path.py
tests/test_permissions_gate.py tests/test_import_cycle_budget.py
tests/test_node_server.py tests/test_node_client.py
tests/test_subagent_acp_remote.py tests/test_acp_remote.py
tests/test_cli_ops_connection_commands.py`
-- all passed on this tree (703 in the run that covered the first
eight groups, the node's own suite and the real-node cases among them).
Covered: each tool on a remote path answers as its local tool would,
through a stood-in link and a real daemon; a path outside the roots is
refused on both sides; a write with a stale digest is refused and
writes nothing; the address form on what the tool says about paths,
and a read's lines, a grep's matches and a diff's body untouched; the
shell's refusal of typed ssh to a registered machine, by host, by
alias, in a pipeline, as `ssh.exe` and `SSH.EXE`, and with a whole
Windows path or a quoted destination or port when the host is Windows,
a copy with scp, sftp or rsync not refused, and the file tools named
only with the lane on; a fetched name holding the
other platform's separators lands as one name under the machine's
directory and nothing lands elsewhere, and a landing that would
resolve outside fetches nothing; a Windows path form read here with the
lane on; the read
permission inside the roots and the question elsewhere, remembered per
machine and directory, a write asked wherever it falls and its grant
kept per file and apart from a read; a read's digest kept per agent,
so another agent's read lets no stale overwrite through and an agent
that never read the file expects nothing; a delivery fetched whole
under a key of its path (two files of one name, two names a
filesystem reads as one, and the percent-code pair all land apart),
refused over the cap and refused outside the roots whatever was read
before.
- `uv run pytest` on this tree after the merge with main -- 31210
passed,
231 skipped, 2 xfailed, 4 failed, every one of the four in a test that
needs what this machine lacks (`test_rpc_files.py`, the two simulation
cases and the kimi-code case that fail the same way on main here); the
groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Diff coverage of this change against main after the merge: 95.27% (805
of
845 executable changed lines), against the 90% gate. The node suite,
the remote-path tests and the real-node cases among the groups above run
on the merged tree.
- Real machines (two registered Linux machines): fifty lines from the
middle of a 5 MB file read through the daemon in 30 to 35 ms, byte for
byte what this computer reads; list, find and grep answered; a write
and an edit landed as the local tools would, and a write after the file
changed was refused; a path outside the roots refused; a file written
by an agent there fetched back by address. The shell's refusals are
covered by the tests above.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint`,
`ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
User docs follow once the lane is no longer debug-only.
## Risk
Behind the debug lane, as the two changes before it. With it on, a
remote
path reaches the machine only through its daemon, inside the roots each
call names and under the read and write questions above; the shell
cannot
run a command on a registered machine by address, and a copy to or from
it
with scp, sftp or rsync is the plain shell's behind exec's own question,
as
on main. A write refused for a stale
digest writes nothing. A delivery fetches at most 20 MB per file into
the
workspace, and only from the directories the agents work in on the
machine. Rollback is a plain revert.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#920 (merged); depends on #921.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 16:10:18 +08:00
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_cli_agent_s_prompt_reaches_its_stdin_as_it_is_and_the_end_of_it_lets_it_leave(tmp_path, agent):
|
|
|
|
|
host = _Host()
|
|
|
|
|
await host.ask(protocol.AGENT_LAUNCH, {"channel": 1, "command": agent, "cwd": str(tmp_path)})
|
|
|
|
|
|
|
|
|
|
prompt = _frame("one") + "\n" + _frame("two", 2) + "\n"
|
|
|
|
|
await host.tell(protocol.AGENT_STDIN, {"channel": 1, "data": prompt, "eof": True})
|
|
|
|
|
|
|
|
|
|
await host.until(lambda: host.notified(protocol.AGENT_EXIT))
|
|
|
|
|
answered = [json.loads(frame["line"])["id"] for frame in host.notified(protocol.AGENT_FRAME)]
|
|
|
|
|
assert answered == [1, 2], "both lines, in order, before the end of its stdin"
|
|
|
|
|
assert host.notified(protocol.AGENT_EXIT) == [{"channel": 1, "rc": 0}], "stdin closed, the agent left on its own"
|
|
|
|
|
assert host.node.agents == {}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_the_end_of_stdin_alone_is_a_frame_too_and_one_for_no_channel_is_dropped(tmp_path, agent):
|
|
|
|
|
host = _Host()
|
|
|
|
|
await host.ask(protocol.AGENT_LAUNCH, {"channel": 1, "command": agent, "cwd": str(tmp_path)})
|
|
|
|
|
|
|
|
|
|
await host.tell(protocol.AGENT_STDIN, {"channel": 9, "data": "nobody", "eof": True})
|
|
|
|
|
await host.tell(protocol.AGENT_STDIN, {"channel": 1, "eof": True})
|
|
|
|
|
|
|
|
|
|
await host.until(lambda: host.notified(protocol.AGENT_EXIT))
|
|
|
|
|
assert host.notified(protocol.AGENT_FRAME) == [] and host.notified(protocol.AGENT_EXIT) == [{"channel": 1, "rc": 0}]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_input_after_the_end_of_stdin_is_dropped_and_the_agent_is_left_alone(tmp_path):
|
|
|
|
|
host = _Host()
|
|
|
|
|
await host.ask(protocol.AGENT_LAUNCH, {"channel": 1, "command": "sleep 2", "cwd": str(tmp_path)})
|
|
|
|
|
|
|
|
|
|
await host.tell(protocol.AGENT_STDIN, {"channel": 1, "eof": True})
|
|
|
|
|
await host.tell(protocol.AGENT_STDIN, {"channel": 1, "data": "late", "eof": True})
|
|
|
|
|
|
|
|
|
|
await host.until(lambda: host.notified(protocol.AGENT_EXIT))
|
|
|
|
|
assert host.notified(protocol.AGENT_EXIT) == [{"channel": 1, "rc": 0}], "ended on its own, not by the late input"
|
|
|
|
|
|
|
|
|
|
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_cancel_ends_the_agent_s_whole_process_group(tmp_path, agent):
|
|
|
|
|
host = _Host()
|
|
|
|
|
started = await host.ask(protocol.AGENT_LAUNCH, {"channel": 2, "command": agent, "cwd": str(tmp_path)})
|
|
|
|
|
pid = started["result"]["pid"]
|
|
|
|
|
await host.tell(protocol.AGENT_FRAME, {"channel": 2, "line": _frame("spawn")})
|
|
|
|
|
await host.until(lambda: host.notified(protocol.AGENT_FRAME))
|
|
|
|
|
child = json.loads(host.notified(protocol.AGENT_FRAME)[0]["line"])["result"]["pid"]
|
|
|
|
|
assert not _gone(pid) and not _gone(child)
|
|
|
|
|
|
|
|
|
|
assert (await host.ask(protocol.AGENT_CANCEL, {"channel": 2}, request_id=3))["result"] == {}
|
|
|
|
|
|
|
|
|
|
await host.until(lambda: host.notified(protocol.AGENT_EXIT))
|
|
|
|
|
assert host.notified(protocol.AGENT_EXIT)[0]["channel"] == 2
|
|
|
|
|
assert host.notified(protocol.AGENT_EXIT)[0]["rc"] < 0, "ended by a signal"
|
|
|
|
|
await host.until(lambda: _gone(pid) and _gone(child))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_cancel_ends_what_the_agent_put_in_a_session_of_its_own(tmp_path, agent):
|
|
|
|
|
host = _Host()
|
|
|
|
|
started = await host.ask(protocol.AGENT_LAUNCH, {"channel": 2, "command": agent, "cwd": str(tmp_path)})
|
|
|
|
|
pid = started["result"]["pid"]
|
|
|
|
|
await host.tell(protocol.AGENT_FRAME, {"channel": 2, "line": _frame("detach")})
|
|
|
|
|
await host.until(lambda: host.notified(protocol.AGENT_FRAME))
|
|
|
|
|
child = json.loads(host.notified(protocol.AGENT_FRAME)[0]["line"])["result"]["pid"]
|
|
|
|
|
assert os.getpgid(child) != os.getpgid(pid), "the child escaped the agent's group"
|
|
|
|
|
|
|
|
|
|
await host.ask(protocol.AGENT_CANCEL, {"channel": 2}, request_id=3)
|
|
|
|
|
|
|
|
|
|
await host.until(lambda: _gone(pid) and _gone(child))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _tree(*, own_sessions: bool) -> tuple[subprocess.Popen[str], int]:
|
|
|
|
|
"""A process with a child under it, each in a session of its own or both in this one."""
|
|
|
|
|
flag = "start_new_session=True" if own_sessions else ""
|
|
|
|
|
parent = subprocess.Popen(
|
|
|
|
|
[
|
|
|
|
|
sys.executable,
|
|
|
|
|
"-c",
|
|
|
|
|
"import subprocess, sys, time; print(subprocess.Popen([sys.executable, '-c',"
|
|
|
|
|
f" 'import time; time.sleep(30)'], {flag}).pid, flush=True); time.sleep(30)",
|
|
|
|
|
],
|
|
|
|
|
start_new_session=own_sessions,
|
|
|
|
|
stdout=subprocess.PIPE,
|
|
|
|
|
text=True,
|
|
|
|
|
)
|
|
|
|
|
assert parent.stdout is not None
|
|
|
|
|
return parent, int(parent.stdout.readline())
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _ends(pid: int) -> bool:
|
|
|
|
|
deadline = time.monotonic() + 10
|
|
|
|
|
while not _gone(pid) and time.monotonic() < deadline:
|
|
|
|
|
time.sleep(0.05)
|
|
|
|
|
return _gone(pid)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _end_quietly(parent: subprocess.Popen[str], child: int) -> None:
|
|
|
|
|
for pid in (parent.pid, child):
|
|
|
|
|
try:
|
|
|
|
|
os.kill(pid, signal.SIGKILL)
|
|
|
|
|
except ProcessLookupError:
|
|
|
|
|
pass
|
|
|
|
|
parent.wait(timeout=10)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_a_tree_is_read_off_ps_and_ended_with_what_left_its_group():
|
|
|
|
|
# A leader of its own session, as the node starts an agent, with a child in
|
|
|
|
|
# a session of its own under it, as Claude Code's shell tool starts a command.
|
|
|
|
|
parent, child = _tree(own_sessions=True)
|
|
|
|
|
try:
|
|
|
|
|
assert os.getpgid(child) != os.getpgid(parent.pid)
|
|
|
|
|
assert server._descendants(parent.pid) == [child]
|
|
|
|
|
assert server._descendants(2**22 - 1) == []
|
|
|
|
|
|
|
|
|
|
server._end_tree(parent.pid)
|
|
|
|
|
|
|
|
|
|
assert parent.wait(timeout=10) != 0
|
|
|
|
|
assert _ends(child), "the child's own session was ended with the tree"
|
|
|
|
|
finally:
|
|
|
|
|
_end_quietly(parent, child)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_a_tree_in_the_node_s_own_group_is_ended_one_by_one_and_the_group_stays():
|
|
|
|
|
parent, child = _tree(own_sessions=False)
|
|
|
|
|
try:
|
|
|
|
|
assert os.getpgid(parent.pid) == os.getpgid(child) == os.getpgrp()
|
|
|
|
|
|
|
|
|
|
server._end_tree(parent.pid)
|
|
|
|
|
|
|
|
|
|
assert parent.wait(timeout=10) != 0
|
|
|
|
|
assert _ends(child)
|
|
|
|
|
finally:
|
|
|
|
|
_end_quietly(parent, child)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_command_not_on_the_machine_and_a_full_node_are_refused_by_code(tmp_path, agent, monkeypatch):
|
|
|
|
|
host = _Host()
|
|
|
|
|
|
|
|
|
|
refused = await host.ask(
|
|
|
|
|
protocol.AGENT_LAUNCH, {"channel": 1, "command": "no-such-agent-for-raven-tests --acp", "cwd": str(tmp_path)}
|
|
|
|
|
)
|
|
|
|
|
assert refused["error"] == {
|
|
|
|
|
"code": protocol.NOT_FOUND,
|
|
|
|
|
"message": "'no-such-agent-for-raven-tests' is not on this machine's PATH",
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(protocol, "MAX_AGENTS", 0)
|
|
|
|
|
full = await host.ask(protocol.AGENT_LAUNCH, {"channel": 1, "command": agent, "cwd": str(tmp_path)})
|
|
|
|
|
assert full["error"] == {"code": protocol.BUSY, "message": "0 agents are already running on this machine"}
|
|
|
|
|
assert host.node.agents == {}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_root_the_machine_cannot_make_refuses_the_launch_with_the_reason(tmp_path, agent):
|
|
|
|
|
host = _Host()
|
|
|
|
|
(tmp_path / "f").write_text("a file\n", encoding="utf-8")
|
|
|
|
|
|
|
|
|
|
refused = await host.ask(protocol.AGENT_LAUNCH, {"channel": 1, "command": agent, "cwd": str(tmp_path / "f" / "r")})
|
|
|
|
|
|
|
|
|
|
assert refused["error"]["code"] == -32603
|
|
|
|
|
assert refused["error"]["message"].startswith(f"could not make {tmp_path / 'f' / 'r'}: ")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
|
|
|
("method", "params", "said"),
|
|
|
|
|
[
|
|
|
|
|
(protocol.AGENT_CANCEL, {"channel": 9}, "no agent on channel 9"),
|
|
|
|
|
(protocol.AGENT_CLOSE, {"channel": "x"}, "channel must be a whole number"),
|
|
|
|
|
(protocol.AGENT_LAUNCH, {"command": "x", "cwd": "/"}, "channel must be a whole number"),
|
|
|
|
|
(protocol.AGENT_LAUNCH, {"channel": 1, "command": "", "cwd": "/"}, "command must be a non-empty string"),
|
|
|
|
|
(
|
|
|
|
|
protocol.AGENT_LAUNCH,
|
|
|
|
|
{"channel": 1, "command": "x 'y", "cwd": "/"},
|
|
|
|
|
"command cannot be parsed: No closing quotation",
|
|
|
|
|
),
|
|
|
|
|
(
|
|
|
|
|
protocol.AGENT_LAUNCH,
|
|
|
|
|
{"channel": 1, "command": "x", "cwd": "/", "env": {"A": 1}},
|
|
|
|
|
"env must be an object of strings",
|
|
|
|
|
),
|
|
|
|
|
(protocol.PROBE_AGENT, {"command": " "}, "command must be a non-empty string"),
|
|
|
|
|
],
|
|
|
|
|
)
|
|
|
|
|
async def test_a_channel_call_that_names_nothing_usable_is_refused_by_name(method, params, said):
|
|
|
|
|
host = _Host()
|
|
|
|
|
|
|
|
|
|
refused = await host.ask(method, params)
|
|
|
|
|
|
|
|
|
|
assert refused["error"] == {"code": -32602, "message": said}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_frame_for_a_channel_nobody_opened_is_dropped():
|
|
|
|
|
host = _Host()
|
|
|
|
|
|
|
|
|
|
await host.tell(protocol.AGENT_FRAME, {"channel": 9, "line": _frame("x")})
|
|
|
|
|
await host.tell(protocol.AGENT_FRAME, {"channel": ["not", "a", "number"], "line": _frame("x")})
|
|
|
|
|
|
|
|
|
|
assert host.frames() == []
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_probe_agent_answers_on_the_path_an_agent_here_would_get(tmp_path, monkeypatch):
|
|
|
|
|
monkeypatch.setattr(server, "capture_login_env", lambda: {"PATH": str(tmp_path)})
|
|
|
|
|
host = _Host()
|
|
|
|
|
found = tmp_path / "fake-agent"
|
|
|
|
|
found.write_text("#!/bin/sh\n", encoding="utf-8")
|
|
|
|
|
found.chmod(0o755)
|
|
|
|
|
|
|
|
|
|
assert (await host.ask(protocol.PROBE_AGENT, {"command": "fake-agent --acp"}))["result"] == {"found": str(found)}
|
|
|
|
|
assert (await host.ask(protocol.PROBE_AGENT, {"command": "nothing-here"}))["result"] == {"found": None}
|
|
|
|
|
overridden = await host.ask(protocol.PROBE_AGENT, {"command": "fake-agent", "env": {"PATH": "/nonexistent"}})
|
|
|
|
|
assert overridden["result"] == {"found": None}, "the entry's own PATH wins, as it does at launch"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_shutdown_ends_every_agent_and_the_loop_leaves_without_an_eof(tmp_path, agent):
|
|
|
|
|
reader = asyncio.StreamReader()
|
|
|
|
|
launch = {"jsonrpc": "2.0", "id": 1, "method": protocol.AGENT_LAUNCH}
|
|
|
|
|
launch["params"] = {"channel": 1, "command": agent, "cwd": str(tmp_path)}
|
|
|
|
|
reader.feed_data(json.dumps(launch).encode() + b"\n")
|
|
|
|
|
reader.feed_data(json.dumps({"jsonrpc": "2.0", "id": 2, "method": protocol.SHUTDOWN}).encode() + b"\n")
|
|
|
|
|
out = io.BytesIO()
|
|
|
|
|
|
|
|
|
|
await asyncio.wait_for(server.serve(reader, out), timeout=30)
|
|
|
|
|
|
|
|
|
|
answers = {
|
|
|
|
|
frame["id"]: frame for frame in (json.loads(line) for line in out.getvalue().splitlines()) if "id" in frame
|
|
|
|
|
}
|
|
|
|
|
assert answers[2]["result"] == {}
|
|
|
|
|
assert _gone(answers[1]["result"]["pid"])
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_requests_are_answered_as_they_finish_not_in_order(monkeypatch):
|
|
|
|
|
async def slow(params: dict[str, Any]) -> dict[str, Any]:
|
|
|
|
|
await asyncio.sleep(0.3)
|
|
|
|
|
return {"text": "slow"}
|
|
|
|
|
|
|
|
|
|
monkeypatch.setitem(server._HANDLERS, "x/slow", slow)
|
|
|
|
|
reader = asyncio.StreamReader()
|
|
|
|
|
reader.feed_data(json.dumps({"jsonrpc": "2.0", "id": 1, "method": "x/slow"}).encode() + b"\n")
|
|
|
|
|
reader.feed_data(json.dumps({"jsonrpc": "2.0", "id": 2, "method": protocol.PING}).encode() + b"\n")
|
|
|
|
|
reader.feed_eof()
|
|
|
|
|
out = io.BytesIO()
|
|
|
|
|
|
|
|
|
|
await server.serve(reader, out)
|
|
|
|
|
|
|
|
|
|
assert [frame["id"] for frame in (json.loads(line) for line in out.getvalue().splitlines())] == [2, 1]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- what the host writes, sends and collects --------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _b64(data: bytes) -> str:
|
|
|
|
|
return base64.b64encode(data).decode("ascii")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_stat_says_what_is_there_and_hashes_a_file_on_request(tree):
|
|
|
|
|
roots = [str(tree)]
|
|
|
|
|
|
|
|
|
|
file = await _result(protocol.STAT, {"path": str(tree / "b.md"), "roots": roots, "hash": True})
|
|
|
|
|
folder = await _result(protocol.STAT, {"path": str(tree / "sub"), "roots": roots})
|
|
|
|
|
missing = await _result(protocol.STAT, {"path": str(tree / "nothing"), "roots": roots})
|
|
|
|
|
|
|
|
|
|
assert file["exists"] and file["kind"] == "file" and file["size"] == 4
|
|
|
|
|
assert file["sha256"] == hashlib.sha256(b"hay\n").hexdigest()
|
|
|
|
|
assert abs(file["mtime"] - (tree / "b.md").stat().st_mtime) < 1e-6
|
|
|
|
|
assert folder["kind"] == "dir" and folder["exists"] and "sha256" not in folder
|
|
|
|
|
assert missing == {"exists": False, "kind": "missing", "size": 0, "mtime": None}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_call_outside_the_roots_is_refused_whatever_it_is(tree):
|
|
|
|
|
outside = str(tree.parent / "secret.txt")
|
|
|
|
|
for method in (protocol.STAT, protocol.PULL):
|
|
|
|
|
code, said = await _refusal(method, {"path": outside, "roots": [str(tree)]})
|
|
|
|
|
assert code == -32602 and "outside allowed directories" in said, method
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_write_through_the_node_is_the_write_the_host_would_have_made(tree):
|
|
|
|
|
path = str(tree / "new" / "note.md")
|
|
|
|
|
roots = [str(tree)]
|
|
|
|
|
|
|
|
|
|
got = await _result(protocol.WRITE, {"path": path, "content": "one\n", "roots": roots})
|
|
|
|
|
appended = await _result(protocol.WRITE, {"path": path, "content": "two\n", "mode": "append", "roots": roots})
|
|
|
|
|
|
|
|
|
|
assert got["text"] == f"Successfully wrote 4 bytes to {path}"
|
|
|
|
|
assert got["file_change"] == {"path": path, "after": "one\n", "before": None}
|
|
|
|
|
assert got["diff"].startswith(f"--- {path}") and "+one" in got["diff"]
|
|
|
|
|
assert appended == {"text": f"Successfully appended 4 bytes to {path}"}
|
|
|
|
|
assert (tree / "new" / "note.md").read_text(encoding="utf-8") == "one\ntwo\n"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_write_that_expects_the_file_as_it_was_read_is_refused_once_it_changed(tree):
|
|
|
|
|
path = str(tree / "b.md")
|
|
|
|
|
roots = [str(tree)]
|
|
|
|
|
read = hashlib.sha256(b"hay\n").hexdigest()
|
|
|
|
|
|
|
|
|
|
first = await _result(protocol.WRITE, {"path": path, "content": "x\n", "expect": read, "roots": roots})
|
|
|
|
|
code, said = await _refusal(protocol.WRITE, {"path": path, "content": "y\n", "expect": read, "roots": roots})
|
|
|
|
|
|
|
|
|
|
assert first["text"].startswith("Successfully wrote")
|
|
|
|
|
assert (code, said) == (protocol.CHANGED, f"{path} changed after it was read; nothing was written")
|
|
|
|
|
assert (tree / "b.md").read_text(encoding="utf-8") == "x\n"
|
|
|
|
|
# A file expected absent that is there, and one expected absent that is.
|
|
|
|
|
assert (await _refusal(protocol.WRITE, {"path": path, "content": "z\n", "expect": "absent", "roots": roots}))[
|
|
|
|
|
0
|
|
|
|
|
] == protocol.CHANGED
|
|
|
|
|
fresh = str(tree / "fresh.md")
|
|
|
|
|
made = await _result(protocol.WRITE, {"path": fresh, "content": "z\n", "expect": "absent", "roots": roots})
|
|
|
|
|
assert made["file_change"]["before"] is None and (tree / "fresh.md").read_text(encoding="utf-8") == "z\n"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_an_edit_through_the_node_is_the_edit_the_host_would_have_made(tree):
|
|
|
|
|
path = str(tree / "sub" / "a.py")
|
|
|
|
|
roots = [str(tree)]
|
|
|
|
|
|
|
|
|
|
got = await _result(
|
|
|
|
|
protocol.EDIT, {"path": path, "old_text": "needle = 1", "new_text": "needle = 2", "roots": roots}
|
|
|
|
|
)
|
|
|
|
|
miss = await _result(
|
|
|
|
|
protocol.EDIT, {"path": path, "old_text": "nothing like this", "new_text": "x", "roots": roots}
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
assert got["text"] == f"Successfully edited {path}"
|
|
|
|
|
assert got["file_change"] == {"path": path, "after": "needle = 2\nhay = 2\n", "before": "needle = 1\nhay = 2\n"}
|
|
|
|
|
assert "-needle = 1" in got["diff"] and "+needle = 2" in got["diff"]
|
|
|
|
|
assert miss["text"].startswith(f"Error: old_text not found in {path}")
|
|
|
|
|
assert (tree / "sub" / "a.py").read_text(encoding="utf-8") == "needle = 2\nhay = 2\n"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_pull_returns_the_bytes_asked_for_with_the_whole_file_s_digest_first(tree):
|
|
|
|
|
data = (tree / "big.txt").read_bytes()
|
|
|
|
|
path = str(tree / "big.txt")
|
|
|
|
|
roots = [str(tree)]
|
|
|
|
|
|
|
|
|
|
first = await _result(protocol.PULL, {"path": path, "offset": 0, "length": 1000, "roots": roots})
|
|
|
|
|
last = await _result(protocol.PULL, {"path": path, "offset": len(data) - 10, "length": 1000, "roots": roots})
|
|
|
|
|
|
|
|
|
|
assert base64.b64decode(first["data"]) == data[:1000] and first["total"] == len(data)
|
|
|
|
|
assert first["sha256"] == hashlib.sha256(data).hexdigest() and first["eof"] is False
|
|
|
|
|
assert base64.b64decode(last["data"]) == data[-10:] and last["eof"] is True and "sha256" not in last
|
|
|
|
|
assert (await _refusal(protocol.PULL, {"path": str(tree / "sub"), "roots": roots}))[1].endswith(
|
|
|
|
|
"is not a file on this machine"
|
|
|
|
|
)
|
|
|
|
|
code, said = await _refusal(protocol.PULL, {"path": path, "length": protocol.CHUNK + 1, "roots": roots})
|
|
|
|
|
assert code == -32602 and f"between 1 and {protocol.CHUNK}" in said
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- tasks that outlive a connection ---------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class _Conn(_Host):
|
|
|
|
|
"""One connection to a daemon several connections share."""
|
|
|
|
|
|
|
|
|
|
def __init__(self, daemon: server.Daemon) -> None:
|
|
|
|
|
self.out = io.BytesIO()
|
|
|
|
|
self.node = server.Node(self.out, daemon)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _record(daemon: server.Daemon, task: str) -> dict[str, Any]:
|
|
|
|
|
return json.loads((daemon.record_dir(task) / "task.json").read_text(encoding="utf-8"))
|
|
|
|
|
|
|
|
|
|
|
feat(*): read, write and deliver files on a registered machine by address (#922)
## Summary
Third of four changes that let a Raven dispatch work to the machines in
its machine registry. #920 is merged; this branch is opened against main
ahead of #921, so the diff here includes #921's commits as well: review
from "read, write and deliver files on a registered machine by address"
on (the commits before it are #921's, carried here until it merges).
What this does:
- The file tools take a path on a registered machine as it is written
for scp -- `user@host:/path`, or `scp://user@host:port//path` when the
port is not 22 -- and answer it through the machine's daemon
(`raven/agent/tools/remote_files.py`, hooked from `filesystem.py` and
`file_search.py`): `read_file`, `list_dir`, `find` and `grep` run
Raven's own tools there and bring back the answer, so a read of fifty
lines of a large file moves those fifty lines; `write_file` and
`edit_file` write there, with the digest of that agent's own last read
-- the conversation's, or its sub-agent run's -- as the expected
state, so a file changed since it read it is refused and nothing is
written, and another agent's read does not stand in for its own. The
path is resolved against the registry
only, never this computer's ssh config, so one path names one machine
on any computer; one that matches no row is refused, not guessed.
- What is shown back as an address is only what the tool says about
paths -- its errors, its headers and footers, a write's account, a
diff's two header lines; a read's numbered lines, a search's matches
and context, and a diff's body come back as they are, so the model
edits against the bytes the file holds. The shell's refusal of typed
`ssh` to a registered machine (`raven/agent/tools/shell.py`,
`machine_exec.py`) reads the program in any spelling -- a path, a
backslash, and on a Windows host `ssh.exe`, `SSH.EXE` or a whole
`C:\...\ssh.exe`, with the backslashes the shell there keeps and
without the quotes it takes off (a quoted piece and a bare one next to
it making one word), so a quoted destination or port is matched like a
bare one -- and, with the lane on, names the file tools. `scp`, `sftp`
and `rsync` to a registered machine stay what they are on main: a copy
starts nothing there and passes exec's own question like any command,
and the file tools cannot carry a directory, a binary file or a large
one, so the copy stays the route for those. A drive-relative or
extended Windows path (`C:file`, `\\?\C:\...`, UNC) is this
computer's, not a machine nobody registered.
- Permissions (`raven/permissions/`): a path inside the directories
Raven's agents work in on that machine -- its session directory, its
`remoteCwd`, the row's listed `paths` -- is read without asking;
elsewhere a read asks once per machine and directory for the session.
A write there asks wherever it falls, as a write here does, and its
grant is for the one file, apart from any read. The daemon checks the
roots again on the machine, links included.
- `deliver_files` fetches one file by its address, whole and checked,
under a 20 MB cap, into `<workspace>/pulled/<machine>/<key>/<name>`,
the key a digest of the file's path on the machine, so two paths never
share a landing: not two files of one name from two directories, and
not two names this computer's filesystem reads as one (`Report.txt`
and `report.txt`, a trailing dot). The name beside the key is written
as one this computer keeps (a backslash, what Windows reserves in a
name, control characters and the percent sign itself as
percent-codes), and a landing that would not resolve under the
machine's delivery directory is refused before anything is made; a
bigger file is named with its size and read there instead. Only a file
under the directories this Raven's agents work in
on that machine is fetched: the tool asks nothing, and a read some
conversation was allowed is no evidence for this one, so a file
elsewhere is refused and read there instead. Nothing reclaims
`pulled/`: a delivery is opened from its landing for as long as its
record lasts, and a bound by size or age is left out of this change on
purpose.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_remote_file_tools.py
tests/test_permissions_remote_read.py tests/test_deliver_files_tool.py
tests/test_shell_machine_channel.py tests/test_ops_remote_path.py
tests/test_permissions_gate.py tests/test_import_cycle_budget.py
tests/test_node_server.py tests/test_node_client.py
tests/test_subagent_acp_remote.py tests/test_acp_remote.py
tests/test_cli_ops_connection_commands.py`
-- all passed on this tree (703 in the run that covered the first
eight groups, the node's own suite and the real-node cases among them).
Covered: each tool on a remote path answers as its local tool would,
through a stood-in link and a real daemon; a path outside the roots is
refused on both sides; a write with a stale digest is refused and
writes nothing; the address form on what the tool says about paths,
and a read's lines, a grep's matches and a diff's body untouched; the
shell's refusal of typed ssh to a registered machine, by host, by
alias, in a pipeline, as `ssh.exe` and `SSH.EXE`, and with a whole
Windows path or a quoted destination or port when the host is Windows,
a copy with scp, sftp or rsync not refused, and the file tools named
only with the lane on; a fetched name holding the
other platform's separators lands as one name under the machine's
directory and nothing lands elsewhere, and a landing that would
resolve outside fetches nothing; a Windows path form read here with the
lane on; the read
permission inside the roots and the question elsewhere, remembered per
machine and directory, a write asked wherever it falls and its grant
kept per file and apart from a read; a read's digest kept per agent,
so another agent's read lets no stale overwrite through and an agent
that never read the file expects nothing; a delivery fetched whole
under a key of its path (two files of one name, two names a
filesystem reads as one, and the percent-code pair all land apart),
refused over the cap and refused outside the roots whatever was read
before.
- `uv run pytest` on this tree after the merge with main -- 31210
passed,
231 skipped, 2 xfailed, 4 failed, every one of the four in a test that
needs what this machine lacks (`test_rpc_files.py`, the two simulation
cases and the kimi-code case that fail the same way on main here); the
groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Diff coverage of this change against main after the merge: 95.27% (805
of
845 executable changed lines), against the 90% gate. The node suite,
the remote-path tests and the real-node cases among the groups above run
on the merged tree.
- Real machines (two registered Linux machines): fifty lines from the
middle of a 5 MB file read through the daemon in 30 to 35 ms, byte for
byte what this computer reads; list, find and grep answered; a write
and an edit landed as the local tools would, and a write after the file
changed was refused; a path outside the roots refused; a file written
by an agent there fetched back by address. The shell's refusals are
covered by the tests above.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint`,
`ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
User docs follow once the lane is no longer debug-only.
## Risk
Behind the debug lane, as the two changes before it. With it on, a
remote
path reaches the machine only through its daemon, inside the roots each
call names and under the read and write questions above; the shell
cannot
run a command on a registered machine by address, and a copy to or from
it
with scp, sftp or rsync is the plain shell's behind exec's own question,
as
on main. A write refused for a stale
digest writes nothing. A delivery fetches at most 20 MB per file into
the
workspace, and only from the directories the agents work in on the
machine. Rollback is a plain revert.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#920 (merged); depends on #921.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 16:10:18 +08:00
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_blank_line_an_agent_prints_is_forwarded_recorded_and_replayed(tmp_path):
|
|
|
|
|
"""A CLI agent's answer is its lines as printed: a blank line is a frame, numbered, kept and replayed like any."""
|
|
|
|
|
daemon = server.Daemon(tmp_path / "node")
|
|
|
|
|
first = _Conn(daemon)
|
|
|
|
|
command = f"{sys.executable} -c \"print('a'); print(); print('b')\""
|
|
|
|
|
await first.ask(
|
|
|
|
|
protocol.AGENT_LAUNCH, {"channel": 1, "task": "t-blank", "command": command, "cwd": str(tmp_path / "w")}
|
|
|
|
|
)
|
|
|
|
|
await first.until(lambda: first.notified(protocol.AGENT_EXIT))
|
|
|
|
|
|
|
|
|
|
assert [(f["line"], f["n"]) for f in first.notified(protocol.AGENT_FRAME)] == [("a", 1), ("", 2), ("b", 3)]
|
|
|
|
|
assert (daemon.record_dir("t-blank") / "frames.jsonl").read_bytes() == b"a\n\nb\n"
|
fix(*): machine runs outlive a killed daemon, a refused failure announce, a dead-link cancel (#957)
## Summary
Seven edges of a run on a registered machine, left by the review of #934
and by timing probes on the real machine. Each is a fix the host or the
node makes on its own; nothing changes for an entry without `machine`.
- **A Raven node killed outright no longer leaves agents behind or a
bare `exit -1`.** The next daemon brings every record left at `running`
to `lost` at its start (`Daemon.recover`), ends the agent when its pid
is still the process it was started as (group leader, the recorded
command, and the start mark the launch recorded: Linux's `/proc` start
tick, elsewhere `ps -o lstart`), and writes why; a record without the
mark, or a pid whose mark differs, is left alone and recorded as gone.
`agent/exit` and the attach answer carry that `reason`; the host hands
it on in place of the exit code (`ChannelEndpoint.exited(rc, reason)`),
so a cli run says "the Raven node on this machine was restarted while
the agent ran, and the agent is gone" and a picked-up acp run says the
same. On Linux the agent is also tied to the daemon's life
(`PR_SET_PDEATHSIG`, a SIGTERM). Protocol number unchanged: the field is
optional and old nodes never send it.
- **A failed remote run's error survives a draining-refused announce.**
The manager's two error branches read whether the announce was taken, as
the reply branch does (`announced = taken is not False`), and the acp
backend keeps the record of a prompt that went out and failed, the cli
backend that of a timed-out command, for the caller that reports it. The
next start reads how the task ended off the machine and announces the
error.
- **A cli run picked up after a restart binds its instance to the
session id in its output** (`_bind_picked_up`), as the dispatch's
deferred commit would have; with no id in the output the reply carries
the same "not resumable" warning the dispatch would. A `provisioned` id
or a stateless run binds nothing.
- **A cancel while the machine's link is known dead returns at once.**
`_cancel_turn` reads `ChannelEndpoint.offline` (parked, or the link not
alive), marks the session unsettled and leaves the held `session/cancel`
for the next link, instead of waiting the settle budget for a settlement
that cannot arrive. In the gap before the heartbeat notices nothing
changes.
- **An openai row's tunnel gives up a taken local port for another.**
The port is checked again just before ssh starts (`_still_free`), and
ssh's own "Address already in use" / "Could not request local
forwarding" ends in another try (`_TUNNEL_TRIES` = 3) rather than in a
request sent to whoever took the port; each attempt reads only what ssh
wrote in its own run of the log.
- **A run record that is not JSON, or not UTF-8, is set aside once**
(`<name>.json.unreadable-<ts>`), named in the log that once, instead of
being warned about at every start; a record a later version wrote is
left alone as before. (The first revision read the record as text and
let a `UnicodeDecodeError` out of the whole scan, found by the Windows
review; the record is read as bytes now and decoded by `json.loads`, so
a decoding failure is set aside like a JSON one, and one bad record
never stops the others from being read.)
- **A path written as remote that names no registered machine** is
refused in one sentence for `write_file` and `edit_file` too, never
taken for a relative path here. This has been the case since the debug
switch went (#934); a regression test pins it.
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Every fix has a test that failed on the old code first.
- `.venv/bin/python -m pytest -n 6 -q tests/test_node_server.py
tests/test_node_client.py tests/test_remote_runs.py
tests/test_subagent_manager.py tests/test_subagent_cli_remote.py
tests/test_remote_file_tools.py tests/test_subagent_acp_remote.py
tests/test_resume_remote.py tests/test_remote_resume.py
tests/test_subagent_acp.py` - all green on macOS (the Linux-only pact
test skipped there); the last run of the four fast files 331 passed, cli
remote 52, remote file tools 24, the acp remote recovery tests 3
- Full suite: 31577 passed, 234 skipped, 2 xfailed in 6 min 14 s; the 3
failures are the ones `main` has on this machine (a host without
LibreOffice, two simulation records). Changed-line coverage: 98.18% of
165 executable changed lines (`scripts/coverage_gate.py diff --base-ref
origin/main`); the 3 uncovered lines are the Linux-only `prctl` import
block, which CI's Linux shards run.
- `ruff check` / `ruff format --check`, `ty check` on the changed
modules, `lint-imports` (10 contracts kept),
`scripts/check_commit_file.py`, `scripts/check_source_language.py`,
`scripts/check_large_files.py`: clean.
- Real machine (ravenx2, an isolated registry copy and Raven home per
tree; the node of each tree installed under its own `node_dir`; the same
probe script on this branch and on `main`):
| Probe | `main` | this branch |
|---|---|---|
| P1 `kill -9` the daemon under a running cli turn | 4.9 s after the
kill: `CLI agent 'x' exited -1`; the loop the daemon had started was
still running (1 left) | 5.1 s: `CLI agent 'x': the Raven node on this
machine was restarted while the agent ran, and the agent is gone`; 0
left; a new daemon pid |
| P2 an acp prompt that fails after it went out, `reported_by_caller` |
the prompt failed (`exit 5`); no record left | record kept; the next
start announced the error with `exit 5` and removed the record |
| P3 cancel while the link is known dead; the cancel on the next link |
the cancel returned after 2.35-2.46 s, held until the link was back (a 2
s backoff in the probe; up to the 5 s budget in production); it landed
on the next link (task frames 3->4) | 0.00 s; landed the same way (3->4
within 0.1 s of the restore) |
| P4 the chosen local tunnel port is taken by a listener | the tunnel
used the taken port; the request timed out against the listener that
held it | the next port; HTTP 200 through the machine |
| P6 a derived-id cli create left running, picked up by the next start |
picked up ok, no binding (`commits=[]`) | picked up ok, the handle bound
to `edge-123` |
After the merge of `main` (which brought #947's MCP streams, #953's
host-death sweep and #963's install fixes into the same files): the
related groups (node server/client/bridge, remote runs, resume, cli
remote, acp remote, manager, remote file tools, acp client, import-cycle
budget) 1006 passed; diff coverage 97.30% of 185 changed lines (the 5
uncovered are the Linux-only `prctl` lines, run by CI's Linux shards);
the real-machine probes re-run on the merged tree: 8/8 PASS (P1 5.2 s, 0
orphans; P2 record kept and the error announced; P3 0.00 s and the
cancel landed on the next link; P4 the next port, HTTP 200; P6 bound to
`edge-123`).
Docs checked against the code before pushing: CONTEXT.md's Raven node
entry (records of failures, set-aside records, the daemon's recovery and
the `reason`), the module docstrings of `remote.tunnel`,
`remote_runs.pending`, `Daemon.recover`, `_still_ours`, `_death_pact`,
`ChannelEndpoint.exited/offline`, `_bind_picked_up`,
`_track_on_machine`, the `agent/exit` comment in `protocol.py`, and the
CHANGELOG entry.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
- [x] Security impact considered - the daemon ends only a pid that is
still the process it started: group leader of its own group, running the
recorded command, and started when the record says (`born`, written at
the launch); a reused pid fails the last mark even for the same program
and is left alone, the record then saying the agent is gone. No new
network surface; the `reason` is the node's fixed sentence, never user
or agent text.
- [x] Backward compatibility considered - `agent/exit`'s `reason` is
optional, protocol number unchanged; a host on this branch reads an old
node's exits as before, an old host ignores the field. Records written
before this branch read unchanged.
- [x] Rollback path is clear for risky changes - a plain revert; nothing
is migrated.
## Related Issues
Follow-up to #934 (the two non-blocking notes of its review) and to the
timing probes run on #921/#922/#934.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-16 10:09:34 +08:00
|
|
|
assert isinstance(_record(daemon, "t-blank")["born"], str), "the record carries when the agent started"
|
feat(*): read, write and deliver files on a registered machine by address (#922)
## Summary
Third of four changes that let a Raven dispatch work to the machines in
its machine registry. #920 is merged; this branch is opened against main
ahead of #921, so the diff here includes #921's commits as well: review
from "read, write and deliver files on a registered machine by address"
on (the commits before it are #921's, carried here until it merges).
What this does:
- The file tools take a path on a registered machine as it is written
for scp -- `user@host:/path`, or `scp://user@host:port//path` when the
port is not 22 -- and answer it through the machine's daemon
(`raven/agent/tools/remote_files.py`, hooked from `filesystem.py` and
`file_search.py`): `read_file`, `list_dir`, `find` and `grep` run
Raven's own tools there and bring back the answer, so a read of fifty
lines of a large file moves those fifty lines; `write_file` and
`edit_file` write there, with the digest of that agent's own last read
-- the conversation's, or its sub-agent run's -- as the expected
state, so a file changed since it read it is refused and nothing is
written, and another agent's read does not stand in for its own. The
path is resolved against the registry
only, never this computer's ssh config, so one path names one machine
on any computer; one that matches no row is refused, not guessed.
- What is shown back as an address is only what the tool says about
paths -- its errors, its headers and footers, a write's account, a
diff's two header lines; a read's numbered lines, a search's matches
and context, and a diff's body come back as they are, so the model
edits against the bytes the file holds. The shell's refusal of typed
`ssh` to a registered machine (`raven/agent/tools/shell.py`,
`machine_exec.py`) reads the program in any spelling -- a path, a
backslash, and on a Windows host `ssh.exe`, `SSH.EXE` or a whole
`C:\...\ssh.exe`, with the backslashes the shell there keeps and
without the quotes it takes off (a quoted piece and a bare one next to
it making one word), so a quoted destination or port is matched like a
bare one -- and, with the lane on, names the file tools. `scp`, `sftp`
and `rsync` to a registered machine stay what they are on main: a copy
starts nothing there and passes exec's own question like any command,
and the file tools cannot carry a directory, a binary file or a large
one, so the copy stays the route for those. A drive-relative or
extended Windows path (`C:file`, `\\?\C:\...`, UNC) is this
computer's, not a machine nobody registered.
- Permissions (`raven/permissions/`): a path inside the directories
Raven's agents work in on that machine -- its session directory, its
`remoteCwd`, the row's listed `paths` -- is read without asking;
elsewhere a read asks once per machine and directory for the session.
A write there asks wherever it falls, as a write here does, and its
grant is for the one file, apart from any read. The daemon checks the
roots again on the machine, links included.
- `deliver_files` fetches one file by its address, whole and checked,
under a 20 MB cap, into `<workspace>/pulled/<machine>/<key>/<name>`,
the key a digest of the file's path on the machine, so two paths never
share a landing: not two files of one name from two directories, and
not two names this computer's filesystem reads as one (`Report.txt`
and `report.txt`, a trailing dot). The name beside the key is written
as one this computer keeps (a backslash, what Windows reserves in a
name, control characters and the percent sign itself as
percent-codes), and a landing that would not resolve under the
machine's delivery directory is refused before anything is made; a
bigger file is named with its size and read there instead. Only a file
under the directories this Raven's agents work in
on that machine is fetched: the tool asks nothing, and a read some
conversation was allowed is no evidence for this one, so a file
elsewhere is refused and read there instead. Nothing reclaims
`pulled/`: a delivery is opened from its landing for as long as its
record lasts, and a bound by size or age is left out of this change on
purpose.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_remote_file_tools.py
tests/test_permissions_remote_read.py tests/test_deliver_files_tool.py
tests/test_shell_machine_channel.py tests/test_ops_remote_path.py
tests/test_permissions_gate.py tests/test_import_cycle_budget.py
tests/test_node_server.py tests/test_node_client.py
tests/test_subagent_acp_remote.py tests/test_acp_remote.py
tests/test_cli_ops_connection_commands.py`
-- all passed on this tree (703 in the run that covered the first
eight groups, the node's own suite and the real-node cases among them).
Covered: each tool on a remote path answers as its local tool would,
through a stood-in link and a real daemon; a path outside the roots is
refused on both sides; a write with a stale digest is refused and
writes nothing; the address form on what the tool says about paths,
and a read's lines, a grep's matches and a diff's body untouched; the
shell's refusal of typed ssh to a registered machine, by host, by
alias, in a pipeline, as `ssh.exe` and `SSH.EXE`, and with a whole
Windows path or a quoted destination or port when the host is Windows,
a copy with scp, sftp or rsync not refused, and the file tools named
only with the lane on; a fetched name holding the
other platform's separators lands as one name under the machine's
directory and nothing lands elsewhere, and a landing that would
resolve outside fetches nothing; a Windows path form read here with the
lane on; the read
permission inside the roots and the question elsewhere, remembered per
machine and directory, a write asked wherever it falls and its grant
kept per file and apart from a read; a read's digest kept per agent,
so another agent's read lets no stale overwrite through and an agent
that never read the file expects nothing; a delivery fetched whole
under a key of its path (two files of one name, two names a
filesystem reads as one, and the percent-code pair all land apart),
refused over the cap and refused outside the roots whatever was read
before.
- `uv run pytest` on this tree after the merge with main -- 31210
passed,
231 skipped, 2 xfailed, 4 failed, every one of the four in a test that
needs what this machine lacks (`test_rpc_files.py`, the two simulation
cases and the kimi-code case that fail the same way on main here); the
groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Diff coverage of this change against main after the merge: 95.27% (805
of
845 executable changed lines), against the 90% gate. The node suite,
the remote-path tests and the real-node cases among the groups above run
on the merged tree.
- Real machines (two registered Linux machines): fifty lines from the
middle of a 5 MB file read through the daemon in 30 to 35 ms, byte for
byte what this computer reads; list, find and grep answered; a write
and an edit landed as the local tools would, and a write after the file
changed was refused; a path outside the roots refused; a file written
by an agent there fetched back by address. The shell's refusals are
covered by the tests above.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint`,
`ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
User docs follow once the lane is no longer debug-only.
## Risk
Behind the debug lane, as the two changes before it. With it on, a
remote
path reaches the machine only through its daemon, inside the roots each
call names and under the read and write questions above; the shell
cannot
run a command on a registered machine by address, and a copy to or from
it
with scp, sftp or rsync is the plain shell's behind exec's own question,
as
on main. A write refused for a stale
digest writes nothing. A delivery fetches at most 20 MB per file into
the
workspace, and only from the directories the agents work in on the
machine. Rollback is a plain revert.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#920 (merged); depends on #921.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 16:10:18 +08:00
|
|
|
|
|
|
|
|
second = _Conn(daemon)
|
|
|
|
|
await second.ask(protocol.AGENT_ATTACH, {"channel": 5, "task": "t-blank", "after": 1}, request_id=2)
|
|
|
|
|
await second.until(lambda: second.notified(protocol.AGENT_EXIT))
|
|
|
|
|
assert [(f["line"], f["n"]) for f in second.notified(protocol.AGENT_FRAME)] == [("", 2), ("b", 3)]
|
|
|
|
|
|
|
|
|
|
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_task_is_recorded_as_it_runs_and_goes_on_when_its_connection_ends(tmp_path, agent):
|
|
|
|
|
daemon = server.Daemon(tmp_path / "node")
|
|
|
|
|
first = _Conn(daemon)
|
|
|
|
|
started = await first.ask(
|
|
|
|
|
protocol.AGENT_LAUNCH, {"channel": 1, "task": "t-one", "command": agent, "cwd": str(tmp_path / "w")}
|
|
|
|
|
)
|
|
|
|
|
assert started["result"]["task"] == "t-one"
|
|
|
|
|
pid = started["result"]["pid"]
|
|
|
|
|
await first.tell(protocol.AGENT_FRAME, {"channel": 1, "line": _frame("one")})
|
|
|
|
|
await first.until(lambda: len(first.notified(protocol.AGENT_FRAME)) == 1)
|
|
|
|
|
assert first.notified(protocol.AGENT_FRAME)[0]["n"] == 1
|
|
|
|
|
assert _record(daemon, "t-one")["state"] == "running"
|
|
|
|
|
|
|
|
|
|
await first.node.detach_all()
|
|
|
|
|
|
|
|
|
|
assert not _gone(pid), "the agent runs on without its connection"
|
|
|
|
|
task = daemon.tasks["t-one"]
|
|
|
|
|
assert task.subscriber is None and daemon.connections == set()
|
|
|
|
|
await task.write(_frame("two", 2))
|
|
|
|
|
await first.until(lambda: task.frames == 2)
|
|
|
|
|
assert (daemon.record_dir("t-one") / "frames.jsonl").read_text(encoding="utf-8").count("\n") == 2
|
|
|
|
|
|
|
|
|
|
second = _Conn(daemon)
|
|
|
|
|
attached = await second.ask(protocol.AGENT_ATTACH, {"channel": 7, "task": "t-one", "after": 1}, request_id=2)
|
|
|
|
|
assert attached["result"]["replayed"] == 1 and attached["result"]["rc"] is None
|
|
|
|
|
assert [f["n"] for f in second.notified(protocol.AGENT_FRAME)] == [2], "only what the host had not read"
|
|
|
|
|
assert second.node.channels[7] is task
|
|
|
|
|
await second.ask(protocol.AGENT_CLOSE, {"channel": 7}, request_id=3)
|
|
|
|
|
await second.until(lambda: second.notified(protocol.AGENT_EXIT))
|
|
|
|
|
assert second.notified(protocol.AGENT_EXIT) == [{"channel": 7, "rc": 0}]
|
|
|
|
|
assert first.notified(protocol.AGENT_EXIT) == [], "the connection that ended hears nothing"
|
|
|
|
|
assert _record(daemon, "t-one")["state"] == "exited" and _record(daemon, "t-one")["rc"] == 0
|
|
|
|
|
assert 7 not in second.node.channels and daemon.running() == []
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_finished_task_is_replayed_from_its_record_by_a_daemon_that_never_ran_it(tmp_path, agent):
|
|
|
|
|
root = tmp_path / "node"
|
|
|
|
|
first = _Conn(server.Daemon(root))
|
|
|
|
|
await first.ask(protocol.AGENT_LAUNCH, {"channel": 1, "task": "t-done", "command": agent, "cwd": str(tmp_path)})
|
|
|
|
|
await first.tell(protocol.AGENT_FRAME, {"channel": 1, "line": _frame("a")})
|
|
|
|
|
await first.tell(protocol.AGENT_FRAME, {"channel": 1, "line": _frame("b", 2)})
|
|
|
|
|
await first.until(lambda: len(first.notified(protocol.AGENT_FRAME)) == 2)
|
|
|
|
|
await first.ask(protocol.AGENT_CLOSE, {"channel": 1}, request_id=2)
|
|
|
|
|
await first.until(lambda: first.notified(protocol.AGENT_EXIT))
|
|
|
|
|
|
|
|
|
|
later = _Conn(server.Daemon(root))
|
|
|
|
|
attached = await later.ask(protocol.AGENT_ATTACH, {"channel": 1, "task": "t-done", "after": 0})
|
|
|
|
|
|
|
|
|
|
assert attached["result"]["replayed"] == 2 and attached["result"]["rc"] == 0
|
|
|
|
|
assert [f["n"] for f in later.notified(protocol.AGENT_FRAME)] == [1, 2]
|
|
|
|
|
assert later.notified(protocol.AGENT_EXIT) == [{"channel": 1, "rc": 0}]
|
|
|
|
|
assert 1 not in later.node.channels, "a finished task holds no channel"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_record_left_running_by_a_daemon_that_died_is_read_as_lost(tmp_path):
|
|
|
|
|
root = tmp_path / "node"
|
|
|
|
|
daemon = server.Daemon(root)
|
|
|
|
|
record = daemon.record_dir("t-gone")
|
|
|
|
|
record.mkdir(parents=True)
|
|
|
|
|
(record / "task.json").write_text(
|
|
|
|
|
json.dumps({"task": "t-gone", "state": "running", "pid": 1, "started": 1.0, "frames": 1}), encoding="utf-8"
|
|
|
|
|
)
|
|
|
|
|
(record / "frames.jsonl").write_text('{"jsonrpc":"2.0","id":1,"result":{}}\n', encoding="utf-8")
|
|
|
|
|
conn = _Conn(daemon)
|
|
|
|
|
|
|
|
|
|
attached = await conn.ask(protocol.AGENT_ATTACH, {"channel": 1, "task": "t-gone"})
|
|
|
|
|
|
|
|
|
|
assert attached["result"]["rc"] == server.LOST_RC and attached["result"]["replayed"] == 1
|
fix(*): machine runs outlive a killed daemon, a refused failure announce, a dead-link cancel (#957)
## Summary
Seven edges of a run on a registered machine, left by the review of #934
and by timing probes on the real machine. Each is a fix the host or the
node makes on its own; nothing changes for an entry without `machine`.
- **A Raven node killed outright no longer leaves agents behind or a
bare `exit -1`.** The next daemon brings every record left at `running`
to `lost` at its start (`Daemon.recover`), ends the agent when its pid
is still the process it was started as (group leader, the recorded
command, and the start mark the launch recorded: Linux's `/proc` start
tick, elsewhere `ps -o lstart`), and writes why; a record without the
mark, or a pid whose mark differs, is left alone and recorded as gone.
`agent/exit` and the attach answer carry that `reason`; the host hands
it on in place of the exit code (`ChannelEndpoint.exited(rc, reason)`),
so a cli run says "the Raven node on this machine was restarted while
the agent ran, and the agent is gone" and a picked-up acp run says the
same. On Linux the agent is also tied to the daemon's life
(`PR_SET_PDEATHSIG`, a SIGTERM). Protocol number unchanged: the field is
optional and old nodes never send it.
- **A failed remote run's error survives a draining-refused announce.**
The manager's two error branches read whether the announce was taken, as
the reply branch does (`announced = taken is not False`), and the acp
backend keeps the record of a prompt that went out and failed, the cli
backend that of a timed-out command, for the caller that reports it. The
next start reads how the task ended off the machine and announces the
error.
- **A cli run picked up after a restart binds its instance to the
session id in its output** (`_bind_picked_up`), as the dispatch's
deferred commit would have; with no id in the output the reply carries
the same "not resumable" warning the dispatch would. A `provisioned` id
or a stateless run binds nothing.
- **A cancel while the machine's link is known dead returns at once.**
`_cancel_turn` reads `ChannelEndpoint.offline` (parked, or the link not
alive), marks the session unsettled and leaves the held `session/cancel`
for the next link, instead of waiting the settle budget for a settlement
that cannot arrive. In the gap before the heartbeat notices nothing
changes.
- **An openai row's tunnel gives up a taken local port for another.**
The port is checked again just before ssh starts (`_still_free`), and
ssh's own "Address already in use" / "Could not request local
forwarding" ends in another try (`_TUNNEL_TRIES` = 3) rather than in a
request sent to whoever took the port; each attempt reads only what ssh
wrote in its own run of the log.
- **A run record that is not JSON, or not UTF-8, is set aside once**
(`<name>.json.unreadable-<ts>`), named in the log that once, instead of
being warned about at every start; a record a later version wrote is
left alone as before. (The first revision read the record as text and
let a `UnicodeDecodeError` out of the whole scan, found by the Windows
review; the record is read as bytes now and decoded by `json.loads`, so
a decoding failure is set aside like a JSON one, and one bad record
never stops the others from being read.)
- **A path written as remote that names no registered machine** is
refused in one sentence for `write_file` and `edit_file` too, never
taken for a relative path here. This has been the case since the debug
switch went (#934); a regression test pins it.
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Every fix has a test that failed on the old code first.
- `.venv/bin/python -m pytest -n 6 -q tests/test_node_server.py
tests/test_node_client.py tests/test_remote_runs.py
tests/test_subagent_manager.py tests/test_subagent_cli_remote.py
tests/test_remote_file_tools.py tests/test_subagent_acp_remote.py
tests/test_resume_remote.py tests/test_remote_resume.py
tests/test_subagent_acp.py` - all green on macOS (the Linux-only pact
test skipped there); the last run of the four fast files 331 passed, cli
remote 52, remote file tools 24, the acp remote recovery tests 3
- Full suite: 31577 passed, 234 skipped, 2 xfailed in 6 min 14 s; the 3
failures are the ones `main` has on this machine (a host without
LibreOffice, two simulation records). Changed-line coverage: 98.18% of
165 executable changed lines (`scripts/coverage_gate.py diff --base-ref
origin/main`); the 3 uncovered lines are the Linux-only `prctl` import
block, which CI's Linux shards run.
- `ruff check` / `ruff format --check`, `ty check` on the changed
modules, `lint-imports` (10 contracts kept),
`scripts/check_commit_file.py`, `scripts/check_source_language.py`,
`scripts/check_large_files.py`: clean.
- Real machine (ravenx2, an isolated registry copy and Raven home per
tree; the node of each tree installed under its own `node_dir`; the same
probe script on this branch and on `main`):
| Probe | `main` | this branch |
|---|---|---|
| P1 `kill -9` the daemon under a running cli turn | 4.9 s after the
kill: `CLI agent 'x' exited -1`; the loop the daemon had started was
still running (1 left) | 5.1 s: `CLI agent 'x': the Raven node on this
machine was restarted while the agent ran, and the agent is gone`; 0
left; a new daemon pid |
| P2 an acp prompt that fails after it went out, `reported_by_caller` |
the prompt failed (`exit 5`); no record left | record kept; the next
start announced the error with `exit 5` and removed the record |
| P3 cancel while the link is known dead; the cancel on the next link |
the cancel returned after 2.35-2.46 s, held until the link was back (a 2
s backoff in the probe; up to the 5 s budget in production); it landed
on the next link (task frames 3->4) | 0.00 s; landed the same way (3->4
within 0.1 s of the restore) |
| P4 the chosen local tunnel port is taken by a listener | the tunnel
used the taken port; the request timed out against the listener that
held it | the next port; HTTP 200 through the machine |
| P6 a derived-id cli create left running, picked up by the next start |
picked up ok, no binding (`commits=[]`) | picked up ok, the handle bound
to `edge-123` |
After the merge of `main` (which brought #947's MCP streams, #953's
host-death sweep and #963's install fixes into the same files): the
related groups (node server/client/bridge, remote runs, resume, cli
remote, acp remote, manager, remote file tools, acp client, import-cycle
budget) 1006 passed; diff coverage 97.30% of 185 changed lines (the 5
uncovered are the Linux-only `prctl` lines, run by CI's Linux shards);
the real-machine probes re-run on the merged tree: 8/8 PASS (P1 5.2 s, 0
orphans; P2 record kept and the error announced; P3 0.00 s and the
cancel landed on the next link; P4 the next port, HTTP 200; P6 bound to
`edge-123`).
Docs checked against the code before pushing: CONTEXT.md's Raven node
entry (records of failures, set-aside records, the daemon's recovery and
the `reason`), the module docstrings of `remote.tunnel`,
`remote_runs.pending`, `Daemon.recover`, `_still_ours`, `_death_pact`,
`ChannelEndpoint.exited/offline`, `_bind_picked_up`,
`_track_on_machine`, the `agent/exit` comment in `protocol.py`, and the
CHANGELOG entry.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
- [x] Security impact considered - the daemon ends only a pid that is
still the process it started: group leader of its own group, running the
recorded command, and started when the record says (`born`, written at
the launch); a reused pid fails the last mark even for the same program
and is left alone, the record then saying the agent is gone. No new
network surface; the `reason` is the node's fixed sentence, never user
or agent text.
- [x] Backward compatibility considered - `agent/exit`'s `reason` is
optional, protocol number unchanged; a host on this branch reads an old
node's exits as before, an old host ignores the field. Records written
before this branch read unchanged.
- [x] Rollback path is clear for risky changes - a plain revert; nothing
is migrated.
## Related Issues
Follow-up to #934 (the two non-blocking notes of its review) and to the
timing probes run on #921/#922/#934.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-16 10:09:34 +08:00
|
|
|
assert attached["result"]["reason"] == server.RESTARTED_AGENT_GONE, "pid 1 is not the agent: it was already gone"
|
|
|
|
|
assert conn.notified(protocol.AGENT_EXIT) == [
|
|
|
|
|
{"channel": 1, "rc": server.LOST_RC, "reason": server.RESTARTED_AGENT_GONE}
|
|
|
|
|
]
|
|
|
|
|
assert (
|
|
|
|
|
_record(daemon, "t-gone")["state"] == "lost"
|
|
|
|
|
and _record(daemon, "t-gone")["reason"] == server.RESTARTED_AGENT_GONE
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_recover_ends_the_agent_a_killed_daemon_left_behind_and_the_record_says_why(tmp_path):
|
|
|
|
|
"""A daemon killed outright leaves its agents writing to pipes nobody reads: the next daemon ends them at its
|
|
|
|
|
start, records why, and a host that asks after the task reads that sentence instead of a bare exit code."""
|
|
|
|
|
root = tmp_path / "node"
|
|
|
|
|
orphan = subprocess.Popen([sys.executable, "-c", "import time; time.sleep(60)"], start_new_session=True)
|
|
|
|
|
daemon = server.Daemon(root)
|
|
|
|
|
record = daemon.record_dir("t-orphan")
|
|
|
|
|
record.mkdir(parents=True)
|
|
|
|
|
(record / "task.json").write_text(
|
|
|
|
|
json.dumps(
|
|
|
|
|
{
|
|
|
|
|
"task": "t-orphan",
|
|
|
|
|
"state": "running",
|
|
|
|
|
"pid": orphan.pid,
|
|
|
|
|
"born": server._born(orphan.pid),
|
|
|
|
|
"command": f"{sys.executable} -c 'import time; time.sleep(60)'",
|
|
|
|
|
"started": 1.0,
|
|
|
|
|
"frames": 0,
|
|
|
|
|
}
|
|
|
|
|
),
|
|
|
|
|
encoding="utf-8",
|
|
|
|
|
)
|
|
|
|
|
(record / "frames.jsonl").write_text("", encoding="utf-8")
|
|
|
|
|
try:
|
|
|
|
|
assert daemon.recover() == 1
|
|
|
|
|
deadline = time.monotonic() + 5
|
|
|
|
|
while orphan.poll() is None and time.monotonic() < deadline:
|
|
|
|
|
await asyncio.sleep(0.05)
|
|
|
|
|
assert orphan.poll() is not None, "the orphan was ended"
|
|
|
|
|
info = _record(daemon, "t-orphan")
|
|
|
|
|
assert (info["state"], info["rc"], info["reason"]) == ("lost", server.LOST_RC, server.RESTARTED_AGENT_ENDED)
|
|
|
|
|
conn = _Conn(daemon)
|
|
|
|
|
attached = await conn.ask(protocol.AGENT_ATTACH, {"channel": 1, "task": "t-orphan"})
|
|
|
|
|
assert attached["result"]["reason"] == server.RESTARTED_AGENT_ENDED
|
|
|
|
|
assert conn.notified(protocol.AGENT_EXIT) == [
|
|
|
|
|
{"channel": 1, "rc": server.LOST_RC, "reason": server.RESTARTED_AGENT_ENDED}
|
|
|
|
|
]
|
|
|
|
|
assert daemon.recover() == 0, "once: a lost record is not brought down again"
|
|
|
|
|
finally:
|
|
|
|
|
if orphan.poll() is None:
|
|
|
|
|
orphan.kill()
|
|
|
|
|
orphan.wait()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_a_record_whose_pid_moved_on_to_another_process_is_left_alone(tmp_path):
|
|
|
|
|
"""The pid of a record names another program now: nothing is signalled, the record says the agent is gone."""
|
|
|
|
|
root = tmp_path / "node"
|
|
|
|
|
daemon = server.Daemon(root)
|
|
|
|
|
record = daemon.record_dir("t-reused")
|
|
|
|
|
record.mkdir(parents=True)
|
|
|
|
|
(record / "task.json").write_text(
|
|
|
|
|
json.dumps(
|
|
|
|
|
{
|
|
|
|
|
"task": "t-reused",
|
|
|
|
|
"state": "running",
|
|
|
|
|
"pid": os.getpid(),
|
|
|
|
|
"command": "some-agent-that-is-not-this-test --serve",
|
|
|
|
|
"started": 1.0,
|
|
|
|
|
"frames": 0,
|
|
|
|
|
}
|
|
|
|
|
),
|
|
|
|
|
encoding="utf-8",
|
|
|
|
|
)
|
|
|
|
|
assert (
|
|
|
|
|
server._still_ours(os.getpid(), "some-agent-that-is-not-this-test --serve", server._born(os.getpid())) is False
|
|
|
|
|
)
|
|
|
|
|
assert daemon.recover() == 1
|
|
|
|
|
assert _record(daemon, "t-reused")["reason"] == server.RESTARTED_AGENT_GONE
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_still_ours_says_no_when_ps_fails_or_disagrees_or_the_command_cannot_be_read(monkeypatch):
|
|
|
|
|
"""Every doubt reads as "not the agent": a pid nothing can be said about is left alone."""
|
|
|
|
|
sleeper = subprocess.Popen([sys.executable, "-c", "import time; time.sleep(30)"], start_new_session=True)
|
|
|
|
|
try:
|
|
|
|
|
command = f"{sys.executable} -c 'import time; time.sleep(30)'"
|
|
|
|
|
born = server._born(sleeper.pid)
|
|
|
|
|
assert isinstance(born, str) and born == server._born(sleeper.pid), "the mark is the same twice over"
|
|
|
|
|
assert server._still_ours(sleeper.pid, command, born) is True, "a session leader, the command, the mark"
|
|
|
|
|
assert server._still_ours(sleeper.pid, command, "tick:1") is False, "the same program, started another time"
|
|
|
|
|
assert server._still_ours(sleeper.pid, command, None) is False, "a record without the mark proves nothing"
|
|
|
|
|
assert server._still_ours(sleeper.pid, "sh -c 'an unbalanced quote", born) is False, (
|
|
|
|
|
"a command shlex cannot read"
|
|
|
|
|
)
|
|
|
|
|
assert server._still_ours(sleeper.pid, "", born) is False, "no command recorded: nothing to match"
|
|
|
|
|
|
|
|
|
|
def failing(*_a, **_k):
|
|
|
|
|
raise OSError("no ps here")
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(server.subprocess, "run", failing)
|
|
|
|
|
assert server._still_ours(sleeper.pid, command, born) is False, "ps cannot be run"
|
|
|
|
|
assert server._born(sleeper.pid) is None or sys.platform == "linux", "no ps, no mark (off Linux)"
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(
|
|
|
|
|
server.subprocess, "run", lambda *_a, **_k: subprocess.CompletedProcess([], 1, stdout="", stderr="")
|
|
|
|
|
)
|
|
|
|
|
assert server._still_ours(sleeper.pid, command, born) is False, "ps names no such process"
|
|
|
|
|
finally:
|
|
|
|
|
sleeper.kill()
|
|
|
|
|
sleeper.wait()
|
|
|
|
|
|
|
|
|
|
|
fix(node): know an orphaned agent by its arguments when argv[0] was exec'd into another image (#984)
## Summary
A Raven node daemon that starts after one was killed under a running
agent ends the orphan only when the pid is still the agent the record
names: `_still_ours` wants a process-group leader, born when the record
says, running the recorded command. The command mark read
`basename(argv[0]) in ps -o command=`. On macOS that is false for the
commonest agent command there is: `/usr/bin/python3` is a stub that
execs `Python.app/Contents/MacOS/Python`, so `ps` shows that image as
argv[0] and `python3` nowhere. Measured 2026-10-16 on a macOS machine
reached over ssh (this computer, registered against itself): the daemon
was killed under a `python3 wait.py` agent, the restarted daemon
recorded the task as gone and told the host so, and the agent ran on for
over a minute until the test ended it by pid. Linux is not affected: the
prctl death pact ends the agent with the daemon before anyone looks
(measured the same day on a Linux machine, agent gone in 4.5 s).
The fix: the command is known by the name it was started under *or* by
its positional arguments in the order they were given. The second
commit, after review, replaced "all arguments joined" with that: npm's
`npx` retitles its own process `npm exec <spec>` (measured on macOS, npm
10.9.8), so for the two npx presets this repository ships neither the
name nor `-y <spec>` is in the line, while the package is -- what such a
program keeps is what it was asked to run, and a flag like `-y` is the
first thing it consumes. The name-only match still counts, so nothing
that matched before stops matching. `ps` is asked with `-ww`, as
`raven/utils/processes.py` and the everos server ask it, because procps
cuts the column to `COLUMNS` even into a pipe and the arguments sit at
the end. A bare name that execs into another image still cannot be told
and is left alone; the docstring says so. Marks one and three (group
leader, birth time) are unchanged, so a reused pid is still refused.
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- New:
`test_still_ours_knows_the_agent_by_its_arguments_when_argv0_was_exec_d_into_another_image`
fakes `ps` the way macOS prints it (image path first, arguments intact),
the real npx line (`npm exec <spec>`, and another package under it,
refused), another program with the same arguments (refused), the
name-only match (still counts, even with the arguments out of order),
the positional order (counts), and asserts `-ww` on every `ps` call.
`test_on_macos_an_agent_started_through_the_system_stub_is_still_ours`
runs the real stub on macOS through `Daemon.recover()` and is skipped
elsewhere. Both red on main.
- `pytest tests/test_node_server.py -n 0`: 117 passed, 1 skipped (on
macOS the darwin test runs).
- Real machines, same day, with this node installed. The npx case on the
Mac, through the node: the adapter's recorded pid shows `npm exec
@agentclientprotocol/claude-agent-acp@0.81.1`, `_still_ours` is True for
it while the daemon is up; with the daemon killed under it the adapter
leaves by itself about 1 s later (its stdin was the daemon's pipe; the
ACP server exits on EOF), so the next daemon rightly records gone and
nothing is left. The script case: daemon killed under a running script
agent on the macOS machine -> the turn ends in 7.3 s with "the Raven
node on this machine was restarted while the agent ran; nothing could
read the agent's output any more, so it was ended" and the agent process
is gone 7.5 s after the kill (before this change: "...and the agent is
gone" while the agent ran on for over a minute); on the Linux machine ->
unchanged, "...and the agent is gone" in 3.5 s, agent dead in 3.7 s (the
death pact ends it first).
- `ty check raven/node`, `lint-imports`, `ruff`: clean (10 import
contracts kept).
- Full suite (`-n 6`, on a laptop shared with other test runs), first
commit: 32044 passed, 5 failed. Three are this machine's standing
failures (a LibreOffice case, two simulation cases). The other two,
`test_a_blank_line_an_agent_prints_is_forwarded_recorded_and_replayed`
and
`test_a_launch_that_quit_on_an_old_node_is_named_with_both_versions`,
pass 3/3 when run on their own on this branch; a parallel run of the
same node test files on unchanged main trips other timing-sensitive
cases instead, so they are load, not this change, which touches
`_still_ours` alone. After the second commit: 32046 passed, 174 skipped,
2 xfailed, 3 failed -- the three standing ones only.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
## Risk
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
The check only ever ends a process that is alive, leads its own group,
started at the recorded time and now also matches the recorded
arguments; the widened mark cannot end anything the old one would have
refused for a reason other than the argv[0] rename. Rollback is
reverting the commit.
## Related Issues
N/A
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-16 19:02:14 +08:00
|
|
|
def test_still_ours_knows_the_agent_by_its_arguments_when_argv0_was_exec_d_into_another_image(monkeypatch):
|
|
|
|
|
"""macOS's ``/usr/bin/python3`` is a stub that execs ``Python.app/Contents/MacOS/Python``: ``ps`` then shows that
|
|
|
|
|
image as argv[0] and ``python3`` nowhere, while the arguments -- the agent's own script and ids -- are intact;
|
|
|
|
|
npm's ``npx`` retitles itself ``npm exec <spec>``, keeping the package and dropping its own name and flags. A
|
|
|
|
|
match on the positional arguments counts, so a restarted daemon still ends the orphan rather than reporting
|
|
|
|
|
it gone."""
|
|
|
|
|
sleeper = subprocess.Popen([sys.executable, "-c", "import time; time.sleep(30)"], start_new_session=True)
|
|
|
|
|
try:
|
|
|
|
|
pid = sleeper.pid
|
|
|
|
|
born = server._born(pid)
|
|
|
|
|
assert isinstance(born, str) and born
|
|
|
|
|
shown = {"line": ""}
|
|
|
|
|
seen: list[list[str]] = []
|
|
|
|
|
real_run = subprocess.run
|
|
|
|
|
|
|
|
|
|
def ps(argv, *a, **k):
|
|
|
|
|
if "lstart=" in argv:
|
|
|
|
|
return subprocess.CompletedProcess(argv, 0, stdout=born.removeprefix("lstart:") + "\n", stderr="")
|
|
|
|
|
if "pgid=,command=" in argv:
|
|
|
|
|
seen.append(list(argv))
|
|
|
|
|
return subprocess.CompletedProcess(argv, 0, stdout=f"{pid} {shown['line']}\n", stderr="")
|
|
|
|
|
return real_run(argv, *a, **k)
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(server.subprocess, "run", ps)
|
|
|
|
|
image = "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python"
|
|
|
|
|
shown["line"] = f"{image} /tmp/w/wait.py mark-1 120"
|
|
|
|
|
assert server._still_ours(pid, "python3 /tmp/w/wait.py mark-1 120", born) is True, (
|
|
|
|
|
"the arguments are the agent's"
|
|
|
|
|
)
|
|
|
|
|
assert server._still_ours(pid, "python3 /tmp/w/wait.py mark-2 120", born) is False, "another agent's arguments"
|
|
|
|
|
assert server._still_ours(pid, "python3", born) is False, (
|
|
|
|
|
"a bare name that execs into another image cannot be told"
|
|
|
|
|
)
|
|
|
|
|
assert all("-ww" in argv for argv in seen), "the line is read whole, whatever COLUMNS says"
|
|
|
|
|
# npm's npx retitles itself: neither "npx" nor "-y" is in the line, the package it was asked to run is
|
|
|
|
|
# (measured 2026-10-16 on macOS, npm 10.9.8: ``npm exec @agentclientprotocol/claude-agent-acp@0.81.1``)
|
|
|
|
|
shown["line"] = "npm exec @agentclientprotocol/claude-agent-acp@0.81.1 "
|
|
|
|
|
assert server._still_ours(pid, "npx -y @agentclientprotocol/claude-agent-acp@0.81.1", born) is True, (
|
|
|
|
|
"the package is what npx kept of its arguments; the flag it consumed is not asked for"
|
|
|
|
|
)
|
|
|
|
|
assert server._still_ours(pid, "npx -y @agentclientprotocol/codex-acp@1.13.1", born) is False, (
|
|
|
|
|
"another package under the same retitled name"
|
|
|
|
|
)
|
|
|
|
|
shown["line"] = "/usr/bin/python3 /tmp/w/wait.py 120 mark-1"
|
|
|
|
|
assert server._still_ours(pid, "python3 /tmp/w/wait.py mark-1 120", born) is True, (
|
|
|
|
|
"the name alone still counts even when the arguments are out of order"
|
|
|
|
|
)
|
|
|
|
|
shown["line"] = "/opt/Python.app/Python /tmp/w/wait.py 120 mark-1"
|
|
|
|
|
assert server._still_ours(pid, "python3 /tmp/w/wait.py mark-1 120", born) is False, (
|
|
|
|
|
"the positional arguments count in their order"
|
|
|
|
|
)
|
|
|
|
|
shown["line"] = "/usr/bin/perl /tmp/other.pl mark-1 120"
|
|
|
|
|
assert server._still_ours(pid, "python3 /tmp/w/wait.py mark-1 120", born) is False, "another program"
|
|
|
|
|
shown["line"] = "python3 /tmp/w/wait.py --changed"
|
|
|
|
|
assert server._still_ours(pid, "python3 /tmp/w/wait.py mark-1 120", born) is True, (
|
|
|
|
|
"the name alone still counts, as it always did"
|
|
|
|
|
)
|
|
|
|
|
finally:
|
|
|
|
|
sleeper.kill()
|
|
|
|
|
sleeper.wait()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.skipif(sys.platform != "darwin", reason="macOS's python3 stub is the case")
|
|
|
|
|
def test_on_macos_an_agent_started_through_the_system_stub_is_still_ours(tmp_path):
|
|
|
|
|
"""The real thing, on a Mac: an agent started as ``python3 <script>`` through the system stub is recognised by a
|
|
|
|
|
daemon that comes after, ended, and recorded as ended rather than gone (measured 2026-10-16 on a macOS machine,
|
|
|
|
|
where the orphan outlived the daemon by over a minute)."""
|
|
|
|
|
script = tmp_path / "wait.py"
|
|
|
|
|
script.write_text("import time\ntime.sleep(60)\n", encoding="utf-8")
|
|
|
|
|
orphan = subprocess.Popen(["/usr/bin/python3", str(script)], start_new_session=True)
|
|
|
|
|
try:
|
|
|
|
|
born = server._born(orphan.pid)
|
|
|
|
|
command = f"python3 {script}"
|
|
|
|
|
assert server._still_ours(orphan.pid, command, born) is True
|
|
|
|
|
root = tmp_path / "node"
|
|
|
|
|
daemon = server.Daemon(root)
|
|
|
|
|
record = daemon.record_dir("t-mac-orphan")
|
|
|
|
|
record.mkdir(parents=True)
|
|
|
|
|
(record / "task.json").write_text(
|
|
|
|
|
json.dumps(
|
|
|
|
|
{
|
|
|
|
|
"task": "t-mac-orphan",
|
|
|
|
|
"state": "running",
|
|
|
|
|
"pid": orphan.pid,
|
|
|
|
|
"born": born,
|
|
|
|
|
"command": command,
|
|
|
|
|
"started": 1.0,
|
|
|
|
|
"frames": 0,
|
|
|
|
|
}
|
|
|
|
|
),
|
|
|
|
|
encoding="utf-8",
|
|
|
|
|
)
|
|
|
|
|
(record / "frames.jsonl").write_text("", encoding="utf-8")
|
|
|
|
|
assert daemon.recover() == 1
|
|
|
|
|
deadline = time.monotonic() + 5
|
|
|
|
|
while orphan.poll() is None and time.monotonic() < deadline:
|
|
|
|
|
time.sleep(0.05)
|
|
|
|
|
assert orphan.poll() is not None, "the orphan was ended"
|
|
|
|
|
assert _record(daemon, "t-mac-orphan")["reason"] == server.RESTARTED_AGENT_ENDED
|
|
|
|
|
finally:
|
|
|
|
|
if orphan.poll() is None:
|
|
|
|
|
orphan.kill()
|
|
|
|
|
orphan.wait()
|
|
|
|
|
|
|
|
|
|
|
fix(*): machine runs outlive a killed daemon, a refused failure announce, a dead-link cancel (#957)
## Summary
Seven edges of a run on a registered machine, left by the review of #934
and by timing probes on the real machine. Each is a fix the host or the
node makes on its own; nothing changes for an entry without `machine`.
- **A Raven node killed outright no longer leaves agents behind or a
bare `exit -1`.** The next daemon brings every record left at `running`
to `lost` at its start (`Daemon.recover`), ends the agent when its pid
is still the process it was started as (group leader, the recorded
command, and the start mark the launch recorded: Linux's `/proc` start
tick, elsewhere `ps -o lstart`), and writes why; a record without the
mark, or a pid whose mark differs, is left alone and recorded as gone.
`agent/exit` and the attach answer carry that `reason`; the host hands
it on in place of the exit code (`ChannelEndpoint.exited(rc, reason)`),
so a cli run says "the Raven node on this machine was restarted while
the agent ran, and the agent is gone" and a picked-up acp run says the
same. On Linux the agent is also tied to the daemon's life
(`PR_SET_PDEATHSIG`, a SIGTERM). Protocol number unchanged: the field is
optional and old nodes never send it.
- **A failed remote run's error survives a draining-refused announce.**
The manager's two error branches read whether the announce was taken, as
the reply branch does (`announced = taken is not False`), and the acp
backend keeps the record of a prompt that went out and failed, the cli
backend that of a timed-out command, for the caller that reports it. The
next start reads how the task ended off the machine and announces the
error.
- **A cli run picked up after a restart binds its instance to the
session id in its output** (`_bind_picked_up`), as the dispatch's
deferred commit would have; with no id in the output the reply carries
the same "not resumable" warning the dispatch would. A `provisioned` id
or a stateless run binds nothing.
- **A cancel while the machine's link is known dead returns at once.**
`_cancel_turn` reads `ChannelEndpoint.offline` (parked, or the link not
alive), marks the session unsettled and leaves the held `session/cancel`
for the next link, instead of waiting the settle budget for a settlement
that cannot arrive. In the gap before the heartbeat notices nothing
changes.
- **An openai row's tunnel gives up a taken local port for another.**
The port is checked again just before ssh starts (`_still_free`), and
ssh's own "Address already in use" / "Could not request local
forwarding" ends in another try (`_TUNNEL_TRIES` = 3) rather than in a
request sent to whoever took the port; each attempt reads only what ssh
wrote in its own run of the log.
- **A run record that is not JSON, or not UTF-8, is set aside once**
(`<name>.json.unreadable-<ts>`), named in the log that once, instead of
being warned about at every start; a record a later version wrote is
left alone as before. (The first revision read the record as text and
let a `UnicodeDecodeError` out of the whole scan, found by the Windows
review; the record is read as bytes now and decoded by `json.loads`, so
a decoding failure is set aside like a JSON one, and one bad record
never stops the others from being read.)
- **A path written as remote that names no registered machine** is
refused in one sentence for `write_file` and `edit_file` too, never
taken for a relative path here. This has been the case since the debug
switch went (#934); a regression test pins it.
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Every fix has a test that failed on the old code first.
- `.venv/bin/python -m pytest -n 6 -q tests/test_node_server.py
tests/test_node_client.py tests/test_remote_runs.py
tests/test_subagent_manager.py tests/test_subagent_cli_remote.py
tests/test_remote_file_tools.py tests/test_subagent_acp_remote.py
tests/test_resume_remote.py tests/test_remote_resume.py
tests/test_subagent_acp.py` - all green on macOS (the Linux-only pact
test skipped there); the last run of the four fast files 331 passed, cli
remote 52, remote file tools 24, the acp remote recovery tests 3
- Full suite: 31577 passed, 234 skipped, 2 xfailed in 6 min 14 s; the 3
failures are the ones `main` has on this machine (a host without
LibreOffice, two simulation records). Changed-line coverage: 98.18% of
165 executable changed lines (`scripts/coverage_gate.py diff --base-ref
origin/main`); the 3 uncovered lines are the Linux-only `prctl` import
block, which CI's Linux shards run.
- `ruff check` / `ruff format --check`, `ty check` on the changed
modules, `lint-imports` (10 contracts kept),
`scripts/check_commit_file.py`, `scripts/check_source_language.py`,
`scripts/check_large_files.py`: clean.
- Real machine (ravenx2, an isolated registry copy and Raven home per
tree; the node of each tree installed under its own `node_dir`; the same
probe script on this branch and on `main`):
| Probe | `main` | this branch |
|---|---|---|
| P1 `kill -9` the daemon under a running cli turn | 4.9 s after the
kill: `CLI agent 'x' exited -1`; the loop the daemon had started was
still running (1 left) | 5.1 s: `CLI agent 'x': the Raven node on this
machine was restarted while the agent ran, and the agent is gone`; 0
left; a new daemon pid |
| P2 an acp prompt that fails after it went out, `reported_by_caller` |
the prompt failed (`exit 5`); no record left | record kept; the next
start announced the error with `exit 5` and removed the record |
| P3 cancel while the link is known dead; the cancel on the next link |
the cancel returned after 2.35-2.46 s, held until the link was back (a 2
s backoff in the probe; up to the 5 s budget in production); it landed
on the next link (task frames 3->4) | 0.00 s; landed the same way (3->4
within 0.1 s of the restore) |
| P4 the chosen local tunnel port is taken by a listener | the tunnel
used the taken port; the request timed out against the listener that
held it | the next port; HTTP 200 through the machine |
| P6 a derived-id cli create left running, picked up by the next start |
picked up ok, no binding (`commits=[]`) | picked up ok, the handle bound
to `edge-123` |
After the merge of `main` (which brought #947's MCP streams, #953's
host-death sweep and #963's install fixes into the same files): the
related groups (node server/client/bridge, remote runs, resume, cli
remote, acp remote, manager, remote file tools, acp client, import-cycle
budget) 1006 passed; diff coverage 97.30% of 185 changed lines (the 5
uncovered are the Linux-only `prctl` lines, run by CI's Linux shards);
the real-machine probes re-run on the merged tree: 8/8 PASS (P1 5.2 s, 0
orphans; P2 record kept and the error announced; P3 0.00 s and the
cancel landed on the next link; P4 the next port, HTTP 200; P6 bound to
`edge-123`).
Docs checked against the code before pushing: CONTEXT.md's Raven node
entry (records of failures, set-aside records, the daemon's recovery and
the `reason`), the module docstrings of `remote.tunnel`,
`remote_runs.pending`, `Daemon.recover`, `_still_ours`, `_death_pact`,
`ChannelEndpoint.exited/offline`, `_bind_picked_up`,
`_track_on_machine`, the `agent/exit` comment in `protocol.py`, and the
CHANGELOG entry.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
- [x] Security impact considered - the daemon ends only a pid that is
still the process it started: group leader of its own group, running the
recorded command, and started when the record says (`born`, written at
the launch); a reused pid fails the last mark even for the same program
and is left alone, the record then saying the agent is gone. No new
network surface; the `reason` is the node's fixed sentence, never user
or agent text.
- [x] Backward compatibility considered - `agent/exit`'s `reason` is
optional, protocol number unchanged; a host on this branch reads an old
node's exits as before, an old host ignores the field. Records written
before this branch read unchanged.
- [x] Rollback path is clear for risky changes - a plain revert; nothing
is migrated.
## Related Issues
Follow-up to #934 (the two non-blocking notes of its review) and to the
timing probes run on #921/#922/#934.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-16 10:09:34 +08:00
|
|
|
def test_born_reads_ps_when_proc_has_nothing_to_say(monkeypatch):
|
|
|
|
|
"""Off Linux there is no ``/proc``; on Linux one that cannot be read is the same case: ``ps`` gives the start time."""
|
|
|
|
|
import builtins
|
|
|
|
|
|
|
|
|
|
real_open = builtins.open
|
|
|
|
|
|
|
|
|
|
def no_proc(path, *args, **kwargs):
|
|
|
|
|
if str(path).startswith("/proc/"):
|
|
|
|
|
raise OSError("no proc here")
|
|
|
|
|
return real_open(path, *args, **kwargs)
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(builtins, "open", no_proc)
|
|
|
|
|
sleeper = subprocess.Popen([sys.executable, "-c", "import time; time.sleep(30)"], start_new_session=True)
|
|
|
|
|
try:
|
|
|
|
|
born = server._born(sleeper.pid)
|
|
|
|
|
assert isinstance(born, str) and born.startswith("lstart:") and born == server._born(sleeper.pid)
|
|
|
|
|
finally:
|
|
|
|
|
sleeper.kill()
|
|
|
|
|
sleeper.wait()
|
|
|
|
|
assert server._born(2**22 + 12345) is None, "a pid nothing runs under: no mark at all"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_recover_passes_over_what_is_not_a_record_directory(tmp_path):
|
|
|
|
|
daemon = server.Daemon(tmp_path / "node")
|
|
|
|
|
(tmp_path / "node" / server.TASKS_DIR / "stray.txt").write_text("not a record", encoding="utf-8")
|
|
|
|
|
assert daemon.recover() == 0
|
|
|
|
|
assert (tmp_path / "node" / server.TASKS_DIR / "stray.txt").exists(), "left where it was"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_death_pact_is_signed_on_linux_only():
|
|
|
|
|
pact = server._death_pact()
|
|
|
|
|
if sys.platform == "linux":
|
|
|
|
|
assert pact["preexec_fn"] is server._sign_death_pact
|
|
|
|
|
else:
|
|
|
|
|
assert pact == {}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.skipif(sys.platform != "linux", reason="PR_SET_PDEATHSIG is Linux's")
|
|
|
|
|
async def test_on_linux_an_agent_goes_when_its_daemon_is_killed_outright(tmp_path):
|
|
|
|
|
"""The pact end to end: a parent that starts a sleeper the daemon's way is killed with SIGKILL; the sleeper goes."""
|
|
|
|
|
script = tmp_path / "parent.py"
|
|
|
|
|
script.write_text(
|
|
|
|
|
"import subprocess, sys, time\n"
|
|
|
|
|
"from raven.node import server\n"
|
|
|
|
|
"child = subprocess.Popen([sys.executable, '-c', 'import time; time.sleep(60)'], "
|
|
|
|
|
"start_new_session=True, **server._death_pact())\n"
|
|
|
|
|
"print(child.pid, flush=True)\n"
|
|
|
|
|
"time.sleep(60)\n",
|
|
|
|
|
encoding="utf-8",
|
|
|
|
|
)
|
|
|
|
|
parent = await asyncio.create_subprocess_exec(sys.executable, str(script), stdout=asyncio.subprocess.PIPE)
|
|
|
|
|
assert parent.stdout is not None
|
|
|
|
|
child_pid = int((await asyncio.wait_for(parent.stdout.readline(), timeout=20)).strip())
|
|
|
|
|
parent.kill()
|
|
|
|
|
await parent.wait()
|
|
|
|
|
deadline = time.monotonic() + 5
|
|
|
|
|
while not _gone(child_pid) and time.monotonic() < deadline:
|
|
|
|
|
await asyncio.sleep(0.05)
|
|
|
|
|
assert _gone(child_pid), "the sleeper went with the parent"
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
|
|
|
("params", "code", "said"),
|
|
|
|
|
[
|
|
|
|
|
({"channel": 1, "task": "t-nobody"}, protocol.UNKNOWN_TASK, "no task 't-nobody' is running or recorded"),
|
|
|
|
|
({"channel": 1, "task": "../etc"}, -32602, "task must be a short name"),
|
|
|
|
|
({"channel": 1, "task": "t-x", "after": -1}, -32602, "after must be a whole number"),
|
|
|
|
|
],
|
|
|
|
|
)
|
|
|
|
|
async def test_an_attach_that_names_nothing_usable_is_refused_by_code(tmp_path, params, code, said):
|
|
|
|
|
conn = _Conn(server.Daemon(tmp_path / "node"))
|
|
|
|
|
|
|
|
|
|
answer = await conn.ask(protocol.AGENT_ATTACH, params)
|
|
|
|
|
|
|
|
|
|
assert answer["error"]["code"] == code and said in answer["error"]["message"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_launch_under_a_running_task_s_id_is_refused_and_the_list_names_every_task(tmp_path, agent):
|
|
|
|
|
daemon = server.Daemon(tmp_path / "node")
|
|
|
|
|
old = daemon.record_dir("t-old")
|
|
|
|
|
old.mkdir(parents=True)
|
|
|
|
|
(old / "task.json").write_text(
|
|
|
|
|
json.dumps({"task": "t-old", "state": "exited", "rc": 0, "started": 1.0, "ended": 2.0, "frames": 3}),
|
|
|
|
|
encoding="utf-8",
|
|
|
|
|
)
|
|
|
|
|
conn = _Conn(daemon)
|
|
|
|
|
await conn.ask(protocol.AGENT_LAUNCH, {"channel": 1, "task": "t-run", "command": agent, "cwd": str(tmp_path)})
|
|
|
|
|
|
|
|
|
|
again = await conn.ask(
|
|
|
|
|
protocol.AGENT_LAUNCH, {"channel": 2, "task": "t-run", "command": agent, "cwd": str(tmp_path)}, request_id=2
|
|
|
|
|
)
|
|
|
|
|
listed = (await conn.ask(protocol.TASK_LIST, {}, request_id=3))["result"]["tasks"]
|
|
|
|
|
hello = (await conn.ask(protocol.HELLO, {"protocol": protocol.PROTOCOL}, request_id=4))["result"]
|
|
|
|
|
|
|
|
|
|
assert again["error"]["code"] == -32602 and "already running; attach to it" in again["error"]["message"]
|
|
|
|
|
assert [(row["task"], row["state"], row["attached"]) for row in listed] == [
|
|
|
|
|
("t-old", "exited", False),
|
|
|
|
|
("t-run", "running", True),
|
|
|
|
|
]
|
|
|
|
|
assert hello["mode"] == "daemon" and hello["tasks"] == 1
|
|
|
|
|
await conn.ask(protocol.AGENT_CANCEL, {"channel": 1}, request_id=5)
|
|
|
|
|
await conn.until(lambda: conn.notified(protocol.AGENT_EXIT))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_sweep_drops_the_records_of_tasks_that_ended_long_ago_and_keeps_the_rest(tmp_path):
|
|
|
|
|
daemon = server.Daemon(tmp_path / "node")
|
|
|
|
|
for name, state, ended in (
|
|
|
|
|
("t-ancient", "exited", time.time() - protocol.TASK_KEEP_S - 60),
|
|
|
|
|
("t-recent", "exited", time.time() - 60),
|
|
|
|
|
("t-still", "running", None),
|
|
|
|
|
):
|
|
|
|
|
record = daemon.record_dir(name)
|
|
|
|
|
record.mkdir(parents=True)
|
|
|
|
|
(record / "task.json").write_text(json.dumps({"task": name, "state": state, "ended": ended}), encoding="utf-8")
|
|
|
|
|
|
|
|
|
|
assert daemon.sweep() == 1
|
|
|
|
|
assert sorted(p.name for p in (tmp_path / "node" / server.TASKS_DIR).iterdir()) == ["t-recent", "t-still"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_task_s_records_are_its_owner_s_alone(tmp_path, agent):
|
|
|
|
|
daemon = server.Daemon(tmp_path / "node")
|
|
|
|
|
conn = _Conn(daemon)
|
|
|
|
|
|
|
|
|
|
started = await conn.ask(
|
|
|
|
|
protocol.AGENT_LAUNCH, {"channel": 1, "task": "t-own", "command": agent, "cwd": str(tmp_path / "w")}
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
try:
|
|
|
|
|
assert started["result"]["task"] == "t-own"
|
|
|
|
|
record = daemon.record_dir("t-own")
|
|
|
|
|
assert (tmp_path / "node" / server.TASKS_DIR).stat().st_mode & 0o777 == 0o700
|
|
|
|
|
assert record.stat().st_mode & 0o777 == 0o700
|
|
|
|
|
assert {p.name: p.stat().st_mode & 0o777 for p in record.iterdir()} == {
|
|
|
|
|
"task.json": 0o600,
|
|
|
|
|
"frames.jsonl": 0o600,
|
|
|
|
|
"stderr.log": 0o600,
|
feat(*): lend the host's MCP servers to an acp agent on a machine through its node (#947)
## Summary
Adds MCP lending for remote ACP agents on top of the machine and
recovery
services already merged in #920, #922 and #934. The branch includes
current
main and its default-on registered-machine behavior.
What this does: an acp entry on a registered machine is handed the MCP
servers its row grants, the way an entry on this computer is, and the
servers themselves stay here with their definitions and keys.
- The agent on the machine runs the node's own bridge as a stdio MCP
server (`python -I -m raven.node --mcp --dir <node dir> --task <task>
--lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte
pumps around a connection to the daemon's socket, no MCP parsed, no
credential held, the standard library only.
- The daemon carries the bridge's frames as a *stream* to the host
holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6,
`raven/node/protocol.py`, `raven/node/server.py`). The daemon tells
which
task a bridge is by the process that connected -- the peer pid off the
socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the
parent chain to the agent the daemon started, which the bridge is under
and no other task's process is; the kernel supplies that pid and a peer
cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the
task it names must be the one the connection is a process of. Two tasks
on
one machine run as one user, so no name or file a bridge could present
would tell them apart; the connecting process does. One stream per
bridge, at most 16 per task; a frame
travels in pieces of at most 256 KiB characters, the last marked
(`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the
daemon relays and keeps nothing. A stream ends with either end, the
host's link or the task, and the other end is told.
- The host opens the endpoints for a machine row exactly as for a row
here
(`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in
the host's sandbox, HTTP auth added here), binds them to the machine,
dispatch and acquired agent task
(`NodePool.lend`/`unlend`) before handing out bridge stanzas. On
`mcp/open`
the host checks the daemon-authenticated task against that binding
before
connecting; it rechecks the grant and link after the socket opens. An
unlend, replacement grant or replaced link cannot authorize that pending
connection. The host then pumps the frames both ways
(`raven/node/client.py`).
`node/hello` names the bridge; a node serving without a daemon
(`--stdio`)
names none, and the dispatch then goes without MCP and says so.
- The reply closes with the servers lent and the machine's node, or with
why none were. Remote lending requires Unix-socket support on the host:
on Windows it is withheld before endpoint setup, the task continues, and
the reply explains why. The roster's "injectable MCP" matches that
support
(`session_mcp_delivered`). The cli lane on a machine still
withholds its servers (the next change); the openai lane has no tools.
Not in this change: the cli lane's MCP file on the machine; a stream
that
survives the link going (a bridge sees its server end, the agent reports
the call failed, and the next dispatch works again); MCP for a run
picked
up after this computer restarted (#934): it reports its calls fail.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Follow-up fix 05eaba67a, after the c9de53947 review:
- The nine tests the Windows review named now pin the host fact through
one
fixture, `unix_socket_host` (`tests/conftest.py`), which sets
`raven.mcp.bridge.unix_sockets_available` to true and re-points every
`raven.*` module that imported it by name; their bodies are unchanged.
- Found complete, not just the nine: the whole suite was run with
`asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted
before collection (a pytest plugin, controller and xdist workers), at
the
merge base `6b938a426` and at `c9de53947`, and the two failure sets
compared. 17 node ids failed at the head and not at the base: the nine,
and eight this change added that run the transport for real (the bridge
module, and two lending tests through the stand-in machine). Those eight
are guarded by host capability instead of pinned: the bridge module
skips
when `sys.platform == "win32" or not unix_sockets_available()`, the two
lending tests carry the existing `skipif(not unix_sockets_available())`.
After the fix the same run fails the same 66 node ids at the head as at
the base: nothing this change adds or touches fails on a host without
unix sockets.
- A bridge whose daemon side goes ends as streams end. The host can
close a
stream while a frame of the agent's is on its way; the daemon writes
`mcp/close` and hangs up, and the frame meets the closed socket, or the
read side is reset first. That exited 1 with `BrokenPipeError` or
`ConnectionResetError`; a connection-level end is now exit 0 with
nothing
said, a frame past the cap still exits 1 with its reason, both pumps are
awaited before the socket closes, and a daemon gone before the stream
opens is one sentence, not a traceback. Four deterministic cases with a
stand-in connection fail on `c9de53947` and pass here.
- The bridge test's stand-in agent outlives its bridge, as an agent
outlives
one of its MCP servers, so the later parts of
`test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins`
no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80
here.
- The focused set (node, MCP, remote, manager and roster files) -- 935
passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
simulation cases, which fail the same way on main on this computer);
diff coverage against main 97.67% (420/430). Real machine, node
reinstalled with this tree in a directory of the check's own: a stdlib
ACP
agent's bridge reached a host-held server and 1 MiB and 8 MiB results
crossed whole, bridge exit 0; nothing was left on the machine.
Follow-up fix c9de53947, after the same-user token finding:
- The daemon no longer uses a token. `agent/launch` mints and writes
none,
`mcp.token` and the `hmac` check are gone, and `Daemon.open_stream`
identifies the bridge by the connecting process instead (`_peer_pid` +
`Daemon.task_of`). The same-user path the reviewer kept open -- a peer
reading another task's `mcp.token` and opening its stream -- is closed,
because identity is the connection the kernel reports, not a readable
secret.
- Tests, each failing on the unfixed tree first: a bridge placed under
task
B that names task A is refused (`the connection is a process of task
'agent@box-2', not 'agent@box-1'`), as are a process under no task, an
unplaced connection, and a task the daemon does not run
(`tests/test_node_server.py`); a real-OS test launches an agent, takes
its
real child, and asserts the child resolves to the task while the test's
own
process resolves to none (runs on Linux in CI); `_peer_pid` is checked
against a real socketpair; the in-process and subprocess bridge tests
now
identify by the connecting process, and a bridge this test spawns
itself,
or another task's agent spawns, never opens the stream.
- Windows withholding now covers a row on this computer too, not only a
machine row: `session_mcp_delivered` drops the `machine and` guard and
answers no whenever `unix_sockets_available()` is false, so a local acp
row is no longer advertised as injectable and no longer raises at
dispatch
on a host without the transport; `_mcp_endpoints` yields nothing before
opening an endpoint and `_mcp_note` explains it. The two tests the
Windows
review named now guard on `unix_sockets_available()`, and the
withholding
test is parameterized over a machine row and a local row.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over the node/MCP/remote suite -- 97.63% changed-line coverage
(412/422).
`ruff`, `ty` on the changed modules, and the import-cycle budget pass.
- Real machine (ravenx2, this head, node reinstalled, digest
fdbe132e76f0):
a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB
and
an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the
connecting-process identity admits the legitimate bridge end to end on
real Linux; the cross-task refusal on Linux is the real-OS test above,
which CI runs there.
Earlier review fixes (4dc5b3e1f, the task-to-grant binding). Its
host-side
binding of each dispatch's endpoints to the acquired agent task is kept;
the
daemon-side token this run verified was since replaced by the
connecting-process
identity above (c9de53947), so the token-named tests and reproduction
below no
longer describe the current tree.
- `uv run --no-sync pytest tests/test_node_client.py
tests/test_node_server.py tests/test_node_bridge.py
tests/test_node_relay.py tests/test_node_bundle.py
tests/test_node_install.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py
tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict
--cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report=
--tb=short` -- 1495 passed and one minimal-node stdio response test
failed;
strict idle accounting also flagged the assembly test. Both cases pass
together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or
timing limits were weakened. The new authorization, revocation and host
compatibility tests all pass.
- The five focused cases failed on the unfixed implementation first: a
different authenticated task selecting another grant, a stream with no
task, each missing Unix socket API, and an astral Unicode chunk over the
node's byte limit. Regressions also cover revocation, grant replacement
and link replacement while an endpoint connection is opening, and a real
remote turn continuing when both host Unix socket APIs are absent.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 97.94% (381/389 changed executable lines); this fix alone is 100%
(40/40). Python lint, type, dependency
and import checks pass; all 10 import contracts hold. The scope checks
for commit messages, source language and file sizes pass. Both RPC
generators pass `--check` after integrating current main.
- A real Linux node with two concurrent tasks and a host loopback
endpoint:
before, the attacker's own task token plus the victim's public grant
names returns the victim endpoint's marker; after, it returns no data
and the endpoint is never reached. Its own grant works, unknown and
revoked grants fail, and the same attack remains denied after the relay
is killed and an ordinary ping reattaches both tasks. A full chunk of
262144 astral characters was lost/corrupted before and arrives intact
after. The final documented source tree was reinstalled and checked
again.
- Real ACP dispatch through the node's bridge reaches a host-held stdio
MCP tool and returns its marker; its grant is removed when the turn
ends.
With the host Unix socket APIs removed to simulate Windows, the remote
agent still completes, receives no MCP, explains the host limitation,
and the host MCP ledger records no extra call. This is not a native
Windows run. Dedicated test tasks, host endpoints and bridge processes
are cleaned up.
- CONTEXT.md, module/field docstrings, both collaboration guides, the
external design notes and this description were checked against the
implementation. Main's no-debug-switch behavior and full-turn recovery
documentation are retained.
Full combined-tree verification: `uv run --no-sync pytest -n 4
--tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure
reproduces on current main in a separate checkout: host CLI color
rendering,
LibreOffice detection and the two simulation cases. No other test fails.
The minimal-node response test that failed in the focused parallel
selection
passes in this full run. Diff coverage: 97.94% (381/389 executable
changed lines).
Earlier author verification:
- `uv run pytest tests/test_node_bridge.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_bundle.py tests/test_node_install.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py`
-- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge
and a stdio node naming none, the token written owner-only beside the
record, a stream opened with the task's token and refused with another
or for a task not running, refused with no host holding the task,
refused past 16 per task, one per bridge connection, the pieces of a
frame crossing both ways as they are and a stranger carrying nothing,
and every way a stream ends (the bridge closing, the host closing, the
bridge's connection ending, the host's link ending, the agent ending)
with the other end told; the bridge as a process against a real daemon
over its socket: a frame to the host and one back whole, a large frame
each way in pieces, the agent closing its stdin ending the stream and
the bridge leaving clean, the host closing the stream ending the bridge,
a wrong token and a missing daemon each refused on stderr with exit 1;
the stand-in machine with the real daemon and the real bridge: an acp
stub there that connects to its `mcpServers` and calls a tool reaches a
stdio server here and answers with its result, the stanza handed to it
naming the node's bridge, the task, the dispatch and the server and
nothing of the server's definition or socket, the ledger here showing
one call, no bridge outliving the dispatch; the reply's note and the
roster predicate for a machine row.
- `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed,
4 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`,
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`
and
`test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`,
which fail
the same way on main on this machine.
- Real machine (a registered Linux machine with Claude Code signed in,
the node reinstalled with the code before and after):
A28, before (the
code of #934's head and its node): the turn closed with "MCP servers
'probe' were not lent", Claude Code said no tool of that name was
connected, and the server here saw no call; after: `PROBE_OK:r2-lend`
in 9.4 s, the server here recorded the call, and the turn closed with
"lent to acp agent ... through the Raven node on machine ...". A34:
after
the turn the machine's process list shows no bridge, a grep of its node
directory finds nothing of the server's command or of this computer's
socket directory, and no endpoint socket is left here. A29: with the
tool switched off here, Claude Code answered UNAVAILABLE and the server
saw no call. A30: a 40 s tool call under way, the bridge visible in the
machine's process list, then cancelled: 3 s later no bridge there and
no socket here. A31: the relay ssh killed during a 20 s call: the turn
ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next
dispatch reached the server again in 7.6 s. A32: five calls in one turn
all reached the server and came back; an 8 MiB answer: the server here
produced it, and a plain ACP agent on the machine (a stdlib stub that
speaks the stanza's bridge directly) received it whole through the node
-- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude
Code itself closed the server on that answer and reported the call
failed: the agent's own cap on a tool result, not the link's.
- `ruff`, the commit-message and PR-body checks -- clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese collaboration guides describe lending,
task-bound authorization,
Windows withholding, and the existing CLI/restart limitations.
## Risk
Registered-machine support is enabled by configuration, without the
removed
trial environment switch. An ACP entry on a machine receives its granted
MCP servers on a Unix-socket-capable host; the host's normal MCP policy
still
applies. This intentionally exposes those tools to the selected remote
agent.
The server definition, credentials and endpoint paths remain on the
host.
The daemon identifies a bridge's task by the connecting process (the
peer
pid the kernel reports, walked to the agent it started), not by any
token or
name the bridge presents, so a peer task sharing the OS account cannot
open
another task's stream; the host then authorizes that task against the
dispatch/server grant. A system that reports no peer pid opens no
stream.
On Windows hosts remote lending is withheld explicitly, without aborting
the
remote task. Protocol 6 remains the MCP wire version; host and node must
also
have the same code digest, so updated nodes must be reinstalled.
Rollback is
a revert; the machine then carries no stream and receives no bridge
stanza.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#934 (merged); #920 and #922 (merged).
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-15 15:37:50 +08:00
|
|
|
}, "the frames hold the agent's output as it was, and the token opens its MCP streams"
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
finally:
|
|
|
|
|
daemon.tasks["t-own"].end()
|
|
|
|
|
await daemon.tasks["t-own"].proc.wait()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_shutdown_from_a_host_leaves_a_daemon_s_tasks_running(tmp_path, agent):
|
|
|
|
|
daemon = server.Daemon(tmp_path / "node")
|
|
|
|
|
reader = asyncio.StreamReader()
|
|
|
|
|
launch = {"jsonrpc": "2.0", "id": 1, "method": protocol.AGENT_LAUNCH}
|
|
|
|
|
launch["params"] = {"channel": 1, "task": "t-keep", "command": agent, "cwd": str(tmp_path)}
|
|
|
|
|
reader.feed_data(json.dumps(launch).encode() + b"\n")
|
|
|
|
|
reader.feed_data(json.dumps({"jsonrpc": "2.0", "id": 2, "method": protocol.SHUTDOWN}).encode() + b"\n")
|
|
|
|
|
out = io.BytesIO()
|
|
|
|
|
|
|
|
|
|
await asyncio.wait_for(server.serve(reader, out, daemon), timeout=30)
|
|
|
|
|
|
|
|
|
|
answers = {f["id"]: f for f in (json.loads(line) for line in out.getvalue().splitlines()) if "id" in f}
|
|
|
|
|
pid = answers[1]["result"]["pid"]
|
|
|
|
|
try:
|
|
|
|
|
assert not _gone(pid) and daemon.tasks["t-keep"].subscriber is None
|
|
|
|
|
finally:
|
|
|
|
|
daemon.tasks["t-keep"].end()
|
|
|
|
|
await daemon.tasks["t-keep"].proc.wait()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_the_daemon_serves_on_its_socket_across_connections_and_leaves_when_idle(tmp_path, agent, monkeypatch):
|
|
|
|
|
monkeypatch.setattr(server, "_IDLE_CHECK_S", 0.1)
|
|
|
|
|
root = tmp_path / "n"
|
|
|
|
|
running = asyncio.create_task(server.run_daemon_async(root, idle_s=0.3))
|
|
|
|
|
sock = server.socket_path(root)
|
|
|
|
|
deadline = time.monotonic() + 10
|
|
|
|
|
while not sock.exists() and time.monotonic() < deadline:
|
|
|
|
|
await asyncio.sleep(0.05)
|
|
|
|
|
|
|
|
|
|
async def ask(
|
|
|
|
|
writer: asyncio.StreamWriter, reader: asyncio.StreamReader, request: dict[str, Any]
|
|
|
|
|
) -> dict[str, Any]:
|
|
|
|
|
writer.write(json.dumps(request).encode() + b"\n")
|
|
|
|
|
await writer.drain()
|
|
|
|
|
while True:
|
|
|
|
|
frame = json.loads(await reader.readline())
|
|
|
|
|
if frame.get("id") == request["id"]:
|
|
|
|
|
return frame
|
|
|
|
|
|
|
|
|
|
reader, writer = await asyncio.open_unix_connection(str(sock))
|
|
|
|
|
hello = await ask(writer, reader, {"jsonrpc": "2.0", "id": 1, "method": protocol.HELLO, "params": {}})
|
|
|
|
|
assert hello["result"]["mode"] == "daemon" and hello["result"]["pid"] == os.getpid()
|
|
|
|
|
launch = {"channel": 1, "task": "t-sock", "command": agent, "cwd": str(tmp_path)}
|
|
|
|
|
started = await ask(writer, reader, {"jsonrpc": "2.0", "id": 2, "method": protocol.AGENT_LAUNCH, "params": launch})
|
|
|
|
|
pid = started["result"]["pid"]
|
|
|
|
|
writer.close()
|
|
|
|
|
await writer.wait_closed()
|
|
|
|
|
await asyncio.sleep(0.2)
|
|
|
|
|
assert not _gone(pid), "the agent survives its connection"
|
|
|
|
|
|
|
|
|
|
reader, writer = await asyncio.open_unix_connection(str(sock))
|
|
|
|
|
attach = {"channel": 3, "task": "t-sock", "after": 0}
|
|
|
|
|
answer = await ask(writer, reader, {"jsonrpc": "2.0", "id": 3, "method": protocol.AGENT_ATTACH, "params": attach})
|
|
|
|
|
assert answer["result"]["rc"] is None
|
|
|
|
|
await ask(writer, reader, {"jsonrpc": "2.0", "id": 4, "method": protocol.AGENT_CANCEL, "params": {"channel": 3}})
|
|
|
|
|
exited = json.loads(await reader.readline())
|
|
|
|
|
assert exited["method"] == protocol.AGENT_EXIT and exited["params"]["channel"] == 3
|
|
|
|
|
writer.close()
|
|
|
|
|
await writer.wait_closed()
|
|
|
|
|
|
|
|
|
|
assert await asyncio.wait_for(running, timeout=10) == 0, "nothing to do: the daemon leaves"
|
|
|
|
|
assert not sock.exists()
|
|
|
|
|
assert _record(server.Daemon(root), "t-sock")["state"] == "exited"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_second_daemon_under_the_same_directory_leaves_at_once(tmp_path, monkeypatch):
|
|
|
|
|
monkeypatch.setattr(server, "_IDLE_CHECK_S", 0.1)
|
|
|
|
|
root = tmp_path / "n"
|
|
|
|
|
first = asyncio.create_task(server.run_daemon_async(root, idle_s=0.5))
|
|
|
|
|
deadline = time.monotonic() + 10
|
|
|
|
|
while not server.socket_path(root).exists() and time.monotonic() < deadline:
|
|
|
|
|
await asyncio.sleep(0.05)
|
|
|
|
|
|
|
|
|
|
assert await asyncio.wait_for(server.run_daemon_async(root), timeout=5) == 0
|
|
|
|
|
assert server.socket_path(root).exists(), "the first daemon's socket is untouched"
|
|
|
|
|
assert await asyncio.wait_for(first, timeout=10) == 0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_socket_s_directory_is_this_user_s_own_or_is_not_used(tmp_path, monkeypatch):
|
|
|
|
|
monkeypatch.setattr(server.tempfile, "gettempdir", lambda: str(tmp_path / "tmp"))
|
|
|
|
|
assert server.socket_path(Path("/srv/node")) == Path("/srv/node") / server.RUN_DIR / f"{bundle.install_name()}.sock"
|
|
|
|
|
sock = server.socket_path(tmp_path / ("n" * 120))
|
|
|
|
|
assert sock.parent == tmp_path / "tmp" / f"raven-node-{os.getuid()}" and sock.suffix == ".sock"
|
|
|
|
|
assert len(os.fsencode(sock)) < len(os.fsencode(tmp_path / ("n" * 120) / server.RUN_DIR))
|
|
|
|
|
|
|
|
|
|
assert server.own_dir(sock.parent) == sock.parent
|
|
|
|
|
assert sock.parent.stat().st_mode & 0o777 == 0o700
|
|
|
|
|
os.chmod(sock.parent, 0o755)
|
|
|
|
|
server.own_dir(sock.parent)
|
|
|
|
|
assert sock.parent.stat().st_mode & 0o777 == 0o700, "this user's, with wider bits: narrowed"
|
|
|
|
|
|
|
|
|
|
(tmp_path / "file").write_text("", encoding="utf-8")
|
|
|
|
|
(tmp_path / "real").mkdir()
|
|
|
|
|
os.chmod(tmp_path / "real", 0o755) # an explicit mode: mkdir's is whatever the umask leaves
|
|
|
|
|
(tmp_path / "link").symlink_to(tmp_path / "real")
|
|
|
|
|
for other in (tmp_path / "file", tmp_path / "link"):
|
|
|
|
|
with pytest.raises(PermissionError, match="is not a directory only this user may open"):
|
|
|
|
|
server.own_dir(other)
|
|
|
|
|
assert server.owned(tmp_path / "file") and server.owned(tmp_path / "nothing-there")
|
|
|
|
|
|
|
|
|
|
me = os.getuid()
|
|
|
|
|
monkeypatch.setattr(server.os, "getuid", lambda: me + 1)
|
|
|
|
|
with pytest.raises(PermissionError, match="is not a directory only this user may open"):
|
|
|
|
|
server.own_dir(tmp_path / "real")
|
|
|
|
|
assert (tmp_path / "real").stat().st_mode & 0o777 == 0o755, "another user's: left as it is"
|
|
|
|
|
assert not server.owned(tmp_path / "file") and server.owned(tmp_path / "nothing-there")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_the_daemon_does_not_listen_beside_a_thing_it_cannot_remove_from_its_socket_s_place(
|
|
|
|
|
tmp_path, monkeypatch
|
|
|
|
|
):
|
|
|
|
|
monkeypatch.setattr(server.tempfile, "gettempdir", lambda: str(tmp_path / "tmp"))
|
|
|
|
|
root = tmp_path / "n"
|
|
|
|
|
sock = server.socket_path(root)
|
|
|
|
|
sock.parent.mkdir(parents=True, exist_ok=True)
|
|
|
|
|
sock.mkdir()
|
|
|
|
|
|
|
|
|
|
with pytest.raises(PermissionError, match="is in the way of the daemon and cannot be removed"):
|
|
|
|
|
await server.run_daemon_async(root, idle_s=0.3)
|
|
|
|
|
|
|
|
|
|
assert sock.is_dir() and not server.pid_path(root).exists()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_whole_file_s_digest_is_taken_off_the_loop_the_heartbeat_is_answered_on(tree, monkeypatch):
|
|
|
|
|
digest = server._sha256
|
|
|
|
|
|
|
|
|
|
def slowly(path: Path) -> str:
|
|
|
|
|
time.sleep(0.4)
|
|
|
|
|
return digest(path)
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(server, "_sha256", slowly)
|
|
|
|
|
read = asyncio.create_task(_result(protocol.READ, {"path": str(tree / "b.md"), "roots": [str(tree)]}))
|
|
|
|
|
await asyncio.sleep(0.05)
|
|
|
|
|
before = time.monotonic()
|
|
|
|
|
|
|
|
|
|
assert await _result(protocol.PING, {"t": 1}) == {"t": 1}
|
|
|
|
|
|
|
|
|
|
assert time.monotonic() - before < 0.3, "answered while the digest was still being taken"
|
|
|
|
|
assert (await read)["sha256"] == hashlib.sha256(b"hay\n").hexdigest()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_read_never_lands_inside_a_write_of_the_same_file(tree, monkeypatch):
|
|
|
|
|
"""The page and the digest a read answers with are one state of the file, whatever a write does meanwhile."""
|
|
|
|
|
digest = server._sha256
|
|
|
|
|
|
|
|
|
|
def slowly(path: Path) -> str:
|
|
|
|
|
time.sleep(0.3)
|
|
|
|
|
return digest(path)
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(server, "_sha256", slowly)
|
|
|
|
|
path = str(tree / "b.md")
|
|
|
|
|
read = asyncio.create_task(_result(protocol.READ, {"path": path, "roots": [str(tree)]}))
|
|
|
|
|
await asyncio.sleep(0.05)
|
|
|
|
|
write = asyncio.create_task(_result(protocol.WRITE, {"path": path, "content": "straw\n", "roots": [str(tree)]}))
|
|
|
|
|
|
|
|
|
|
page, written = await asyncio.gather(read, write)
|
|
|
|
|
|
|
|
|
|
assert page["text"].startswith("1| hay") and page["sha256"] == hashlib.sha256(b"hay\n").hexdigest()
|
|
|
|
|
assert written["ok"] and (tree / "b.md").read_text(encoding="utf-8") == "straw\n", "the write landed after"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_a_writer_that_waits_goes_before_new_readers():
|
|
|
|
|
gate = server._FileGate()
|
|
|
|
|
first_in = threading.Event()
|
|
|
|
|
writer_waiting = threading.Event()
|
|
|
|
|
writer_done = threading.Event()
|
|
|
|
|
order: list[str] = []
|
|
|
|
|
|
|
|
|
|
def first_reader() -> None:
|
|
|
|
|
with gate.reading():
|
|
|
|
|
first_in.set()
|
|
|
|
|
writer_waiting.wait(5)
|
|
|
|
|
time.sleep(0.1)
|
|
|
|
|
order.append("first reader out")
|
|
|
|
|
|
|
|
|
|
def writer() -> None:
|
|
|
|
|
writer_waiting.set()
|
|
|
|
|
with gate.writing():
|
|
|
|
|
order.append("writer")
|
|
|
|
|
writer_done.set()
|
|
|
|
|
|
|
|
|
|
def second_reader() -> None:
|
|
|
|
|
with gate.reading():
|
|
|
|
|
order.append("second reader")
|
|
|
|
|
|
|
|
|
|
threads = [
|
|
|
|
|
threading.Thread(target=first_reader),
|
|
|
|
|
threading.Thread(target=writer),
|
|
|
|
|
threading.Thread(target=second_reader),
|
|
|
|
|
]
|
|
|
|
|
threads[0].start()
|
|
|
|
|
first_in.wait(5)
|
|
|
|
|
threads[1].start()
|
|
|
|
|
writer_waiting.wait(5)
|
|
|
|
|
time.sleep(0.02)
|
|
|
|
|
threads[2].start()
|
|
|
|
|
for thread in threads:
|
|
|
|
|
thread.join(5)
|
|
|
|
|
|
|
|
|
|
assert order == ["first reader out", "writer", "second reader"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_page_read_of_a_large_file_keeps_the_heartbeat_answered(tmp_path):
|
|
|
|
|
"""The page is read a chunk at a time, so no one step holds the interpreter for the whole file."""
|
|
|
|
|
big = tmp_path / "big.log"
|
|
|
|
|
with big.open("wb") as handle:
|
|
|
|
|
line = b"2026-10-13 12:00:00 one line of a log that goes on for a while, as log lines do\n"
|
|
|
|
|
for _ in range(120 * 1024 * 1024 // len(line)):
|
|
|
|
|
handle.write(line)
|
|
|
|
|
read = asyncio.create_task(_result(protocol.READ, {"path": str(big), "limit": 3, "roots": [str(tmp_path)]}))
|
|
|
|
|
gaps: list[float] = []
|
|
|
|
|
last = time.monotonic()
|
|
|
|
|
while not read.done():
|
|
|
|
|
assert await _result(protocol.PING, {"t": 1}) == {"t": 1}
|
|
|
|
|
now = time.monotonic()
|
|
|
|
|
gaps.append(now - last)
|
|
|
|
|
last = now
|
|
|
|
|
await asyncio.sleep(0.005)
|
|
|
|
|
|
|
|
|
|
assert max(gaps) < 0.25, f"a ping waited {max(gaps):.2f}s on the read"
|
|
|
|
|
assert (await read)["text"].startswith("1| 2026-10-13 12:00:00 one line")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize("argv", [["--serve"], ["--attach"]])
|
|
|
|
|
def test_serve_and_attach_need_the_node_directory(argv, capsys):
|
|
|
|
|
with pytest.raises(SystemExit):
|
|
|
|
|
node_main.main(argv)
|
|
|
|
|
assert "--dir" in capsys.readouterr().err
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_long_path_socket_is_this_install_s_as_the_short_one_is(tmp_path, monkeypatch):
|
|
|
|
|
monkeypatch.setattr(server.tempfile, "gettempdir", lambda: str(tmp_path / "tmp"))
|
|
|
|
|
root = tmp_path / ("n" * 120)
|
|
|
|
|
monkeypatch.setattr(bundle, "install_name", lambda: "0.9.0-aaaaaaaaaaaa")
|
|
|
|
|
one = server.socket_path(root)
|
|
|
|
|
monkeypatch.setattr(bundle, "install_name", lambda: "0.9.0-bbbbbbbbbbbb")
|
|
|
|
|
other = server.socket_path(root)
|
|
|
|
|
|
|
|
|
|
assert one.parent == other.parent and one != other, "two installs on one long directory listen apart"
|
|
|
|
|
assert server.socket_path(root) == other, "and a relay of the same install finds its daemon"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_launch_in_progress_counts_toward_the_limit(tmp_path, agent, monkeypatch):
|
|
|
|
|
"""Two launches that pass the check before either has registered its task: only one may spawn."""
|
|
|
|
|
host = _Host()
|
|
|
|
|
monkeypatch.setattr(protocol, "MAX_AGENTS", 1)
|
|
|
|
|
hold = threading.Event()
|
|
|
|
|
monkeypatch.setattr(server, "capture_login_env", lambda: (hold.wait(10), dict(os.environ))[1])
|
|
|
|
|
|
|
|
|
|
async def launch(channel: int) -> dict[str, Any]:
|
|
|
|
|
params = {"channel": channel, "task": f"t{channel}", "command": agent, "cwd": str(tmp_path)}
|
|
|
|
|
return await host.ask(protocol.AGENT_LAUNCH, params, request_id=channel)
|
|
|
|
|
|
|
|
|
|
first = asyncio.create_task(launch(1))
|
|
|
|
|
await asyncio.sleep(0.05) # past the check, waiting for the login environment
|
|
|
|
|
second = asyncio.create_task(launch(2))
|
|
|
|
|
await asyncio.sleep(0.05)
|
|
|
|
|
hold.set()
|
|
|
|
|
one, two = await asyncio.gather(first, second)
|
|
|
|
|
|
|
|
|
|
assert "result" in one, one
|
|
|
|
|
assert two["error"] == {"code": protocol.BUSY, "message": "1 agents are already running on this machine"}
|
|
|
|
|
assert len(host.node.daemon.running()) == 1
|
|
|
|
|
await host.ask(protocol.AGENT_CANCEL, {"channel": 1}, request_id=3)
|
|
|
|
|
await host.until(lambda: host.node.daemon.running() == [])
|
feat(*): lend the host's MCP servers to an acp agent on a machine through its node (#947)
## Summary
Adds MCP lending for remote ACP agents on top of the machine and
recovery
services already merged in #920, #922 and #934. The branch includes
current
main and its default-on registered-machine behavior.
What this does: an acp entry on a registered machine is handed the MCP
servers its row grants, the way an entry on this computer is, and the
servers themselves stay here with their definitions and keys.
- The agent on the machine runs the node's own bridge as a stdio MCP
server (`python -I -m raven.node --mcp --dir <node dir> --task <task>
--lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte
pumps around a connection to the daemon's socket, no MCP parsed, no
credential held, the standard library only.
- The daemon carries the bridge's frames as a *stream* to the host
holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6,
`raven/node/protocol.py`, `raven/node/server.py`). The daemon tells
which
task a bridge is by the process that connected -- the peer pid off the
socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the
parent chain to the agent the daemon started, which the bridge is under
and no other task's process is; the kernel supplies that pid and a peer
cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the
task it names must be the one the connection is a process of. Two tasks
on
one machine run as one user, so no name or file a bridge could present
would tell them apart; the connecting process does. One stream per
bridge, at most 16 per task; a frame
travels in pieces of at most 256 KiB characters, the last marked
(`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the
daemon relays and keeps nothing. A stream ends with either end, the
host's link or the task, and the other end is told.
- The host opens the endpoints for a machine row exactly as for a row
here
(`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in
the host's sandbox, HTTP auth added here), binds them to the machine,
dispatch and acquired agent task
(`NodePool.lend`/`unlend`) before handing out bridge stanzas. On
`mcp/open`
the host checks the daemon-authenticated task against that binding
before
connecting; it rechecks the grant and link after the socket opens. An
unlend, replacement grant or replaced link cannot authorize that pending
connection. The host then pumps the frames both ways
(`raven/node/client.py`).
`node/hello` names the bridge; a node serving without a daemon
(`--stdio`)
names none, and the dispatch then goes without MCP and says so.
- The reply closes with the servers lent and the machine's node, or with
why none were. Remote lending requires Unix-socket support on the host:
on Windows it is withheld before endpoint setup, the task continues, and
the reply explains why. The roster's "injectable MCP" matches that
support
(`session_mcp_delivered`). The cli lane on a machine still
withholds its servers (the next change); the openai lane has no tools.
Not in this change: the cli lane's MCP file on the machine; a stream
that
survives the link going (a bridge sees its server end, the agent reports
the call failed, and the next dispatch works again); MCP for a run
picked
up after this computer restarted (#934): it reports its calls fail.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Follow-up fix 05eaba67a, after the c9de53947 review:
- The nine tests the Windows review named now pin the host fact through
one
fixture, `unix_socket_host` (`tests/conftest.py`), which sets
`raven.mcp.bridge.unix_sockets_available` to true and re-points every
`raven.*` module that imported it by name; their bodies are unchanged.
- Found complete, not just the nine: the whole suite was run with
`asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted
before collection (a pytest plugin, controller and xdist workers), at
the
merge base `6b938a426` and at `c9de53947`, and the two failure sets
compared. 17 node ids failed at the head and not at the base: the nine,
and eight this change added that run the transport for real (the bridge
module, and two lending tests through the stand-in machine). Those eight
are guarded by host capability instead of pinned: the bridge module
skips
when `sys.platform == "win32" or not unix_sockets_available()`, the two
lending tests carry the existing `skipif(not unix_sockets_available())`.
After the fix the same run fails the same 66 node ids at the head as at
the base: nothing this change adds or touches fails on a host without
unix sockets.
- A bridge whose daemon side goes ends as streams end. The host can
close a
stream while a frame of the agent's is on its way; the daemon writes
`mcp/close` and hangs up, and the frame meets the closed socket, or the
read side is reset first. That exited 1 with `BrokenPipeError` or
`ConnectionResetError`; a connection-level end is now exit 0 with
nothing
said, a frame past the cap still exits 1 with its reason, both pumps are
awaited before the socket closes, and a daemon gone before the stream
opens is one sentence, not a traceback. Four deterministic cases with a
stand-in connection fail on `c9de53947` and pass here.
- The bridge test's stand-in agent outlives its bridge, as an agent
outlives
one of its MCP servers, so the later parts of
`test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins`
no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80
here.
- The focused set (node, MCP, remote, manager and roster files) -- 935
passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
simulation cases, which fail the same way on main on this computer);
diff coverage against main 97.67% (420/430). Real machine, node
reinstalled with this tree in a directory of the check's own: a stdlib
ACP
agent's bridge reached a host-held server and 1 MiB and 8 MiB results
crossed whole, bridge exit 0; nothing was left on the machine.
Follow-up fix c9de53947, after the same-user token finding:
- The daemon no longer uses a token. `agent/launch` mints and writes
none,
`mcp.token` and the `hmac` check are gone, and `Daemon.open_stream`
identifies the bridge by the connecting process instead (`_peer_pid` +
`Daemon.task_of`). The same-user path the reviewer kept open -- a peer
reading another task's `mcp.token` and opening its stream -- is closed,
because identity is the connection the kernel reports, not a readable
secret.
- Tests, each failing on the unfixed tree first: a bridge placed under
task
B that names task A is refused (`the connection is a process of task
'agent@box-2', not 'agent@box-1'`), as are a process under no task, an
unplaced connection, and a task the daemon does not run
(`tests/test_node_server.py`); a real-OS test launches an agent, takes
its
real child, and asserts the child resolves to the task while the test's
own
process resolves to none (runs on Linux in CI); `_peer_pid` is checked
against a real socketpair; the in-process and subprocess bridge tests
now
identify by the connecting process, and a bridge this test spawns
itself,
or another task's agent spawns, never opens the stream.
- Windows withholding now covers a row on this computer too, not only a
machine row: `session_mcp_delivered` drops the `machine and` guard and
answers no whenever `unix_sockets_available()` is false, so a local acp
row is no longer advertised as injectable and no longer raises at
dispatch
on a host without the transport; `_mcp_endpoints` yields nothing before
opening an endpoint and `_mcp_note` explains it. The two tests the
Windows
review named now guard on `unix_sockets_available()`, and the
withholding
test is parameterized over a machine row and a local row.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over the node/MCP/remote suite -- 97.63% changed-line coverage
(412/422).
`ruff`, `ty` on the changed modules, and the import-cycle budget pass.
- Real machine (ravenx2, this head, node reinstalled, digest
fdbe132e76f0):
a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB
and
an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the
connecting-process identity admits the legitimate bridge end to end on
real Linux; the cross-task refusal on Linux is the real-OS test above,
which CI runs there.
Earlier review fixes (4dc5b3e1f, the task-to-grant binding). Its
host-side
binding of each dispatch's endpoints to the acquired agent task is kept;
the
daemon-side token this run verified was since replaced by the
connecting-process
identity above (c9de53947), so the token-named tests and reproduction
below no
longer describe the current tree.
- `uv run --no-sync pytest tests/test_node_client.py
tests/test_node_server.py tests/test_node_bridge.py
tests/test_node_relay.py tests/test_node_bundle.py
tests/test_node_install.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py
tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict
--cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report=
--tb=short` -- 1495 passed and one minimal-node stdio response test
failed;
strict idle accounting also flagged the assembly test. Both cases pass
together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or
timing limits were weakened. The new authorization, revocation and host
compatibility tests all pass.
- The five focused cases failed on the unfixed implementation first: a
different authenticated task selecting another grant, a stream with no
task, each missing Unix socket API, and an astral Unicode chunk over the
node's byte limit. Regressions also cover revocation, grant replacement
and link replacement while an endpoint connection is opening, and a real
remote turn continuing when both host Unix socket APIs are absent.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 97.94% (381/389 changed executable lines); this fix alone is 100%
(40/40). Python lint, type, dependency
and import checks pass; all 10 import contracts hold. The scope checks
for commit messages, source language and file sizes pass. Both RPC
generators pass `--check` after integrating current main.
- A real Linux node with two concurrent tasks and a host loopback
endpoint:
before, the attacker's own task token plus the victim's public grant
names returns the victim endpoint's marker; after, it returns no data
and the endpoint is never reached. Its own grant works, unknown and
revoked grants fail, and the same attack remains denied after the relay
is killed and an ordinary ping reattaches both tasks. A full chunk of
262144 astral characters was lost/corrupted before and arrives intact
after. The final documented source tree was reinstalled and checked
again.
- Real ACP dispatch through the node's bridge reaches a host-held stdio
MCP tool and returns its marker; its grant is removed when the turn
ends.
With the host Unix socket APIs removed to simulate Windows, the remote
agent still completes, receives no MCP, explains the host limitation,
and the host MCP ledger records no extra call. This is not a native
Windows run. Dedicated test tasks, host endpoints and bridge processes
are cleaned up.
- CONTEXT.md, module/field docstrings, both collaboration guides, the
external design notes and this description were checked against the
implementation. Main's no-debug-switch behavior and full-turn recovery
documentation are retained.
Full combined-tree verification: `uv run --no-sync pytest -n 4
--tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure
reproduces on current main in a separate checkout: host CLI color
rendering,
LibreOffice detection and the two simulation cases. No other test fails.
The minimal-node response test that failed in the focused parallel
selection
passes in this full run. Diff coverage: 97.94% (381/389 executable
changed lines).
Earlier author verification:
- `uv run pytest tests/test_node_bridge.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_bundle.py tests/test_node_install.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py`
-- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge
and a stdio node naming none, the token written owner-only beside the
record, a stream opened with the task's token and refused with another
or for a task not running, refused with no host holding the task,
refused past 16 per task, one per bridge connection, the pieces of a
frame crossing both ways as they are and a stranger carrying nothing,
and every way a stream ends (the bridge closing, the host closing, the
bridge's connection ending, the host's link ending, the agent ending)
with the other end told; the bridge as a process against a real daemon
over its socket: a frame to the host and one back whole, a large frame
each way in pieces, the agent closing its stdin ending the stream and
the bridge leaving clean, the host closing the stream ending the bridge,
a wrong token and a missing daemon each refused on stderr with exit 1;
the stand-in machine with the real daemon and the real bridge: an acp
stub there that connects to its `mcpServers` and calls a tool reaches a
stdio server here and answers with its result, the stanza handed to it
naming the node's bridge, the task, the dispatch and the server and
nothing of the server's definition or socket, the ledger here showing
one call, no bridge outliving the dispatch; the reply's note and the
roster predicate for a machine row.
- `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed,
4 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`,
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`
and
`test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`,
which fail
the same way on main on this machine.
- Real machine (a registered Linux machine with Claude Code signed in,
the node reinstalled with the code before and after):
A28, before (the
code of #934's head and its node): the turn closed with "MCP servers
'probe' were not lent", Claude Code said no tool of that name was
connected, and the server here saw no call; after: `PROBE_OK:r2-lend`
in 9.4 s, the server here recorded the call, and the turn closed with
"lent to acp agent ... through the Raven node on machine ...". A34:
after
the turn the machine's process list shows no bridge, a grep of its node
directory finds nothing of the server's command or of this computer's
socket directory, and no endpoint socket is left here. A29: with the
tool switched off here, Claude Code answered UNAVAILABLE and the server
saw no call. A30: a 40 s tool call under way, the bridge visible in the
machine's process list, then cancelled: 3 s later no bridge there and
no socket here. A31: the relay ssh killed during a 20 s call: the turn
ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next
dispatch reached the server again in 7.6 s. A32: five calls in one turn
all reached the server and came back; an 8 MiB answer: the server here
produced it, and a plain ACP agent on the machine (a stdlib stub that
speaks the stanza's bridge directly) received it whole through the node
-- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude
Code itself closed the server on that answer and reported the call
failed: the agent's own cap on a tool result, not the link's.
- `ruff`, the commit-message and PR-body checks -- clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese collaboration guides describe lending,
task-bound authorization,
Windows withholding, and the existing CLI/restart limitations.
## Risk
Registered-machine support is enabled by configuration, without the
removed
trial environment switch. An ACP entry on a machine receives its granted
MCP servers on a Unix-socket-capable host; the host's normal MCP policy
still
applies. This intentionally exposes those tools to the selected remote
agent.
The server definition, credentials and endpoint paths remain on the
host.
The daemon identifies a bridge's task by the connecting process (the
peer
pid the kernel reports, walked to the agent it started), not by any
token or
name the bridge presents, so a peer task sharing the OS account cannot
open
another task's stream; the host then authorizes that task against the
dispatch/server grant. A system that reports no peer pid opens no
stream.
On Windows hosts remote lending is withheld explicitly, without aborting
the
remote task. Protocol 6 remains the MCP wire version; host and node must
also
have the same code digest, so updated nodes must be reinstalled.
Rollback is
a revert; the machine then carries no stream and receives no bridge
stanza.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#934 (merged); #920 and #922 (merged).
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-15 15:37:50 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
# ---- MCP lent across the link: streams -------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class _Peer:
|
|
|
|
|
"""One connection to a shared daemon, as the host's or a bridge's: every line the daemon wrote to it, decoded.
|
|
|
|
|
|
|
|
|
|
``peer`` is what the system says at the socket: the pid of the process
|
|
|
|
|
that connected, which is how the daemon tells whose bridge it is.
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
def __init__(self, daemon: server.Daemon, *, peer: int | None = None) -> None:
|
|
|
|
|
self.out = io.BytesIO()
|
|
|
|
|
self.node = server.Node(self.out, daemon, peer=peer)
|
|
|
|
|
|
|
|
|
|
def frames(self) -> list[dict[str, Any]]:
|
|
|
|
|
return [json.loads(line) for line in self.out.getvalue().splitlines() if line.strip()]
|
|
|
|
|
|
|
|
|
|
def notified(self, method: str) -> list[dict[str, Any]]:
|
|
|
|
|
return [frame["params"] for frame in self.frames() if frame.get("method") == method]
|
|
|
|
|
|
|
|
|
|
async def ask(self, method: str, params: dict[str, Any], request_id: int = 1) -> dict[str, Any]:
|
|
|
|
|
answer = await server.handle(
|
|
|
|
|
{"jsonrpc": "2.0", "id": request_id, "method": method, "params": params}, self.node
|
|
|
|
|
)
|
|
|
|
|
assert answer is not None
|
|
|
|
|
return answer
|
|
|
|
|
|
|
|
|
|
async def tell(self, method: str, params: dict[str, Any]) -> None:
|
|
|
|
|
assert await server.handle({"jsonrpc": "2.0", "method": method, "params": params}, self.node) is None
|
|
|
|
|
|
|
|
|
|
async def until(self, check: Any, *, timeout: float = 10.0) -> None:
|
|
|
|
|
deadline = time.monotonic() + timeout
|
|
|
|
|
while not check():
|
|
|
|
|
assert time.monotonic() < deadline, "did not happen in time"
|
|
|
|
|
await asyncio.sleep(0.02)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture
|
|
|
|
|
def lineage(monkeypatch):
|
|
|
|
|
"""The process table as the daemon reads it, made up: which pid is under which, by parent."""
|
|
|
|
|
table: dict[int, int] = {}
|
|
|
|
|
monkeypatch.setattr(server, "_process_table", lambda: dict(table))
|
|
|
|
|
return table
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def _launched(
|
|
|
|
|
daemon: server.Daemon, host: _Peer, tmp_path: Path, task: str = "agent@box-1", *, channel: int = 1
|
|
|
|
|
) -> int:
|
|
|
|
|
"""An agent of the host's on the daemon, under ``task``: its pid, which every bridge of the task is under."""
|
|
|
|
|
started = await host.ask(
|
|
|
|
|
protocol.AGENT_LAUNCH, {"channel": channel, "task": task, "command": "sleep 30", "cwd": str(tmp_path)}
|
|
|
|
|
)
|
|
|
|
|
assert started["result"]["task"] == task
|
|
|
|
|
return started["result"]["pid"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _bridge(daemon: server.Daemon, lineage: dict[int, int], *, under: int) -> _Peer:
|
|
|
|
|
"""A bridge's connection, from a process the daemon's process table puts under ``under``."""
|
|
|
|
|
pid = 900_000 + len(lineage)
|
|
|
|
|
lineage[pid] = under
|
|
|
|
|
return _Peer(daemon, peer=pid)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _opening(*, task: str = "agent@box-1", server_name: str = "pg", lend: str = "d1") -> dict[str, Any]:
|
|
|
|
|
return {"task": task, "server": server_name, "lend": lend}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture
|
|
|
|
|
async def daemon(tmp_path: Path):
|
|
|
|
|
d = server.Daemon(tmp_path / "node")
|
|
|
|
|
yield d
|
|
|
|
|
await d.end_all()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_hello_names_the_bridge_an_agent_runs_to_reach_a_lent_server_and_a_stdio_node_names_none(tmp_path):
|
|
|
|
|
with_daemon = server.Node(io.BytesIO(), server.Daemon(tmp_path / "node"))
|
|
|
|
|
answer = await with_daemon.hello({"protocol": protocol.PROTOCOL})
|
|
|
|
|
assert answer["bridge"] == [sys.executable, "-I", "-m", "raven.node", "--mcp", "--dir", str(tmp_path / "node")]
|
|
|
|
|
assert "bridge" not in await server.Node(io.BytesIO()).hello({"protocol": protocol.PROTOCOL})
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_stream_is_opened_for_a_process_under_the_task_s_agent_as_the_system_shows_it(tmp_path, daemon):
|
|
|
|
|
"""The real process table: a child of the agent the daemon started may open the task's stream; this test may not."""
|
|
|
|
|
host = _Peer(daemon)
|
|
|
|
|
started = await host.ask(
|
|
|
|
|
protocol.AGENT_LAUNCH,
|
|
|
|
|
{"channel": 1, "task": "agent@box-1", "command": "sh -c 'sleep 30; true'", "cwd": str(tmp_path)},
|
|
|
|
|
)
|
|
|
|
|
agent = started["result"]["pid"]
|
|
|
|
|
await host.until(lambda: server._descendants(agent))
|
|
|
|
|
(child,) = server._descendants(agent)
|
|
|
|
|
|
|
|
|
|
assert daemon.task_of(child) is daemon.tasks["agent@box-1"] and daemon.task_of(agent) is daemon.tasks["agent@box-1"]
|
|
|
|
|
assert daemon.task_of(os.getpid()) is None, "the daemon's own process, and everything above it, is no task's"
|
|
|
|
|
opened = await _Peer(daemon, peer=child).ask(protocol.MCP_OPEN, _opening())
|
|
|
|
|
assert opened["result"] == {"stream": 1}
|
|
|
|
|
stranger = await _Peer(daemon, peer=os.getpid()).ask(protocol.MCP_OPEN, _opening())
|
|
|
|
|
assert stranger["error"]["code"] == protocol.STREAM_REFUSED
|
|
|
|
|
assert stranger["error"]["message"] == "the connection is not a process of any task on this machine"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
def test_the_peer_of_a_socket_is_the_process_that_connected_and_nothing_where_the_system_does_not_say(monkeypatch):
|
|
|
|
|
left, right = socket.socketpair(socket.AF_UNIX, socket.SOCK_STREAM)
|
|
|
|
|
try:
|
|
|
|
|
assert server._peer_pid(left) == os.getpid() == server._peer_pid(right)
|
|
|
|
|
finally:
|
|
|
|
|
left.close()
|
|
|
|
|
right.close()
|
|
|
|
|
|
|
|
|
|
class _Mute:
|
|
|
|
|
def getsockopt(self, *args: Any) -> bytes:
|
|
|
|
|
raise OSError("not a socket")
|
|
|
|
|
|
|
|
|
|
assert server._peer_pid(_Mute()) is None
|
|
|
|
|
monkeypatch.setattr(server.sys, "platform", "sunos5")
|
|
|
|
|
monkeypatch.delattr(server.socket, "SO_PEERCRED", raising=False)
|
|
|
|
|
assert server._peer_pid(left) is None, "a system that answers neither question places no bridge"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_task_is_found_up_the_parent_chain_only_and_a_chain_that_loops_or_ends_finds_none(
|
|
|
|
|
tmp_path, daemon, lineage
|
|
|
|
|
):
|
|
|
|
|
host = _Peer(daemon)
|
|
|
|
|
agent = await _launched(daemon, host, tmp_path)
|
|
|
|
|
task = daemon.tasks["agent@box-1"]
|
|
|
|
|
lineage.update({agent: 1, 7001: agent, 7002: 7001, 7003: 7004, 7004: 7003, 7005: 1})
|
|
|
|
|
|
|
|
|
|
assert daemon.task_of(agent) is task, "the agent itself"
|
|
|
|
|
assert daemon.task_of(7002) is task, "two steps under it"
|
|
|
|
|
assert daemon.task_of(7003) is None and daemon.task_of(7005) is None and daemon.task_of(0) is None
|
|
|
|
|
assert daemon.task_of(1) is None, "init is nobody's agent"
|
|
|
|
|
lineage.clear()
|
|
|
|
|
assert daemon.task_of(agent) is task, "with no table to read, the agent's own process still is"
|
|
|
|
|
assert daemon.task_of(7001) is None, "and nothing under it can be placed"
|
|
|
|
|
await host.ask(protocol.AGENT_CANCEL, {"channel": 1}, request_id=2)
|
|
|
|
|
await host.until(lambda: host.notified(protocol.AGENT_EXIT))
|
|
|
|
|
assert daemon.task_of(agent) is None, "a task that ended is no task"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_stream_is_refused_for_a_process_of_another_task_of_no_task_or_one_the_system_cannot_place(
|
|
|
|
|
tmp_path, daemon, lineage
|
|
|
|
|
):
|
|
|
|
|
host = _Peer(daemon)
|
|
|
|
|
mine = await _launched(daemon, host, tmp_path)
|
|
|
|
|
theirs = await _launched(daemon, host, tmp_path, "agent@box-2", channel=2)
|
|
|
|
|
|
|
|
|
|
other = await _bridge(daemon, lineage, under=theirs).ask(protocol.MCP_OPEN, _opening())
|
|
|
|
|
assert other["error"]["code"] == protocol.STREAM_REFUSED
|
|
|
|
|
assert other["error"]["message"] == "the connection is a process of task 'agent@box-2', not 'agent@box-1'"
|
|
|
|
|
nobody = await _Peer(daemon, peer=424242).ask(protocol.MCP_OPEN, _opening())
|
|
|
|
|
assert (
|
|
|
|
|
nobody["error"]["code"] == protocol.STREAM_REFUSED and "not a process of any task" in nobody["error"]["message"]
|
|
|
|
|
)
|
|
|
|
|
unplaced = await _Peer(daemon, peer=None).ask(protocol.MCP_OPEN, _opening())
|
|
|
|
|
assert unplaced["error"]["code"] == protocol.STREAM_REFUSED
|
|
|
|
|
assert unplaced["error"]["message"] == "this system does not say which process a bridge is, so no stream is opened"
|
|
|
|
|
unknown = await _bridge(daemon, lineage, under=mine).ask(protocol.MCP_OPEN, _opening(task="nobody@box-9"))
|
|
|
|
|
assert unknown["error"]["code"] == protocol.STREAM_REFUSED and "not running" in unknown["error"]["message"]
|
|
|
|
|
assert host.notified(protocol.MCP_OPEN) == [], "a refused stream is not told to the host"
|
|
|
|
|
|
|
|
|
|
bridge = _bridge(daemon, lineage, under=mine)
|
|
|
|
|
opened = await bridge.ask(protocol.MCP_OPEN, _opening())
|
|
|
|
|
|
|
|
|
|
assert opened["result"] == {"stream": 1}
|
|
|
|
|
assert host.notified(protocol.MCP_OPEN) == [{"task": "agent@box-1", "stream": 1, "server": "pg", "lend": "d1"}]
|
|
|
|
|
again = await bridge.ask(protocol.MCP_OPEN, _opening(), request_id=2)
|
|
|
|
|
assert again["error"]["code"] == -32602 and "already" in again["error"]["message"], "one stream per bridge"
|
|
|
|
|
|
|
|
|
|
# The other task's own bridge opens its own stream beside it, told to the host as that task's.
|
|
|
|
|
own = await _bridge(daemon, lineage, under=theirs).ask(protocol.MCP_OPEN, _opening(task="agent@box-2"))
|
|
|
|
|
assert own["result"] == {"stream": 2}
|
|
|
|
|
assert host.notified(protocol.MCP_OPEN)[-1] == {"task": "agent@box-2", "stream": 2, "server": "pg", "lend": "d1"}
|
|
|
|
|
assert [s.task.id for s in daemon.streams.values()] == ["agent@box-1", "agent@box-2"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_stream_needs_a_host_holding_the_task_and_a_task_holds_a_bounded_number(tmp_path, daemon, lineage):
|
|
|
|
|
host = _Peer(daemon)
|
|
|
|
|
agent = await _launched(daemon, host, tmp_path)
|
|
|
|
|
await host.node.detach_all()
|
|
|
|
|
refused = await _bridge(daemon, lineage, under=agent).ask(protocol.MCP_OPEN, _opening())
|
|
|
|
|
assert refused["error"]["code"] == protocol.STREAM_REFUSED and "no host holds" in refused["error"]["message"]
|
|
|
|
|
|
|
|
|
|
back = _Peer(daemon)
|
|
|
|
|
await back.ask(protocol.AGENT_ATTACH, {"channel": 1, "task": "agent@box-1", "after": 0})
|
|
|
|
|
bridges = [_bridge(daemon, lineage, under=agent) for _ in range(protocol.MAX_STREAMS + 1)]
|
|
|
|
|
answers = [await b.ask(protocol.MCP_OPEN, _opening(server_name=f"s{i}")) for i, b in enumerate(bridges)]
|
|
|
|
|
|
|
|
|
|
assert all("result" in a for a in answers[: protocol.MAX_STREAMS])
|
|
|
|
|
assert answers[-1]["error"]["code"] == protocol.STREAM_REFUSED and "open already" in answers[-1]["error"]["message"]
|
|
|
|
|
assert len(daemon.tasks["agent@box-1"].streams) == protocol.MAX_STREAMS
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_the_pieces_of_a_frame_cross_the_daemon_both_ways_as_they_are(tmp_path, daemon, lineage):
|
|
|
|
|
host = _Peer(daemon)
|
|
|
|
|
agent = await _launched(daemon, host, tmp_path)
|
|
|
|
|
bridge = _bridge(daemon, lineage, under=agent)
|
|
|
|
|
stream = (await bridge.ask(protocol.MCP_OPEN, _opening()))["result"]["stream"]
|
|
|
|
|
|
|
|
|
|
await bridge.tell(protocol.MCP_FRAME, {"stream": stream, "text": '{"jsonrpc":"2.0","id":1,', "end": False})
|
|
|
|
|
await bridge.tell(protocol.MCP_FRAME, {"stream": stream, "text": '"method":"tools/list"}', "end": True})
|
|
|
|
|
await host.tell(protocol.MCP_FRAME, {"stream": stream, "text": '{"jsonrpc":"2.0","id":1,"result":{}}', "end": True})
|
|
|
|
|
|
|
|
|
|
assert host.notified(protocol.MCP_FRAME) == [
|
|
|
|
|
{"stream": stream, "text": '{"jsonrpc":"2.0","id":1,', "end": False},
|
|
|
|
|
{"stream": stream, "text": '"method":"tools/list"}', "end": True},
|
|
|
|
|
]
|
|
|
|
|
assert bridge.notified(protocol.MCP_FRAME) == [
|
|
|
|
|
{"stream": stream, "text": '{"jsonrpc":"2.0","id":1,"result":{}}', "end": True}
|
|
|
|
|
]
|
|
|
|
|
stranger = _Peer(daemon)
|
|
|
|
|
await stranger.tell(protocol.MCP_FRAME, {"stream": stream, "text": "x", "end": True})
|
|
|
|
|
assert len(host.notified(protocol.MCP_FRAME)) == 2 and len(bridge.notified(protocol.MCP_FRAME)) == 1, (
|
|
|
|
|
"a connection that is neither end carries nothing"
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_stream_ends_with_either_end_or_the_agent_and_the_other_end_is_told(tmp_path, daemon, lineage):
|
|
|
|
|
host = _Peer(daemon)
|
|
|
|
|
agent = await _launched(daemon, host, tmp_path)
|
|
|
|
|
|
|
|
|
|
# The bridge closes: the host hears it, the stream is gone.
|
|
|
|
|
bridge = _bridge(daemon, lineage, under=agent)
|
|
|
|
|
first = (await bridge.ask(protocol.MCP_OPEN, _opening()))["result"]["stream"]
|
|
|
|
|
await bridge.tell(protocol.MCP_CLOSE, {"stream": first, "reason": "the agent closed the server"})
|
|
|
|
|
assert host.notified(protocol.MCP_CLOSE) == [
|
|
|
|
|
{"stream": first, "task": "agent@box-1", "reason": "the agent closed the server"}
|
|
|
|
|
]
|
|
|
|
|
assert first not in daemon.streams and daemon.tasks["agent@box-1"].streams == {}
|
|
|
|
|
|
|
|
|
|
# The host closes: the bridge hears it.
|
|
|
|
|
bridge = _bridge(daemon, lineage, under=agent)
|
|
|
|
|
second = (await bridge.ask(protocol.MCP_OPEN, _opening()))["result"]["stream"]
|
|
|
|
|
await host.tell(protocol.MCP_CLOSE, {"stream": second, "reason": "the dispatch ended"})
|
|
|
|
|
assert bridge.notified(protocol.MCP_CLOSE) == [
|
|
|
|
|
{"stream": second, "task": "agent@box-1", "reason": "the dispatch ended"}
|
|
|
|
|
]
|
|
|
|
|
|
|
|
|
|
# The bridge's connection ends without a word: the host is told.
|
|
|
|
|
bridge = _bridge(daemon, lineage, under=agent)
|
|
|
|
|
third = (await bridge.ask(protocol.MCP_OPEN, _opening()))["result"]["stream"]
|
|
|
|
|
await bridge.node.detach_all()
|
|
|
|
|
assert host.notified(protocol.MCP_CLOSE)[-1] == {
|
|
|
|
|
"stream": third,
|
|
|
|
|
"task": "agent@box-1",
|
|
|
|
|
"reason": "the bridge ended",
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
# The host's link ends: its streams go, the bridges are told.
|
|
|
|
|
bridge = _bridge(daemon, lineage, under=agent)
|
|
|
|
|
fourth = (await bridge.ask(protocol.MCP_OPEN, _opening()))["result"]["stream"]
|
|
|
|
|
await host.node.detach_all()
|
|
|
|
|
assert bridge.notified(protocol.MCP_CLOSE) == [
|
|
|
|
|
{"stream": fourth, "task": "agent@box-1", "reason": "the host's link ended"}
|
|
|
|
|
]
|
|
|
|
|
assert daemon.streams == {}
|
|
|
|
|
|
|
|
|
|
# The agent ends: the streams of its task end with it.
|
|
|
|
|
back = _Peer(daemon)
|
|
|
|
|
await back.ask(protocol.AGENT_ATTACH, {"channel": 1, "task": "agent@box-1", "after": 0})
|
|
|
|
|
bridge = _bridge(daemon, lineage, under=agent)
|
|
|
|
|
fifth = (await bridge.ask(protocol.MCP_OPEN, _opening()))["result"]["stream"]
|
|
|
|
|
await back.ask(protocol.AGENT_CANCEL, {"channel": 1}, request_id=2)
|
|
|
|
|
await back.until(lambda: back.notified(protocol.AGENT_EXIT))
|
|
|
|
|
await back.until(lambda: bridge.notified(protocol.MCP_CLOSE))
|
|
|
|
|
assert bridge.notified(protocol.MCP_CLOSE) == [
|
|
|
|
|
{"stream": fifth, "task": "agent@box-1", "reason": "the agent ended"}
|
|
|
|
|
]
|
|
|
|
|
assert daemon.streams == {}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_stream_is_refused_when_its_opening_is_malformed_or_the_host_cannot_be_told(tmp_path, daemon, lineage):
|
|
|
|
|
host = _Peer(daemon)
|
|
|
|
|
agent = await _launched(daemon, host, tmp_path)
|
|
|
|
|
bridge = _bridge(daemon, lineage, under=agent)
|
|
|
|
|
|
|
|
|
|
bare = await bridge.ask(protocol.MCP_OPEN, {"task": "agent@box-1", "server": "", "lend": "d1"})
|
|
|
|
|
assert bare["error"]["code"] == -32602 and "non-empty strings" in bare["error"]["message"]
|
|
|
|
|
|
|
|
|
|
host.out.close() # the host's way out is gone, and nobody has noticed yet
|
|
|
|
|
untold = await bridge.ask(protocol.MCP_OPEN, _opening(), request_id=2)
|
|
|
|
|
|
|
|
|
|
assert untold["error"]["code"] == protocol.STREAM_REFUSED and "could not be told" in untold["error"]["message"]
|
|
|
|
|
assert daemon.streams == {} and bridge.node.stream is None and daemon.tasks["agent@box-1"].streams == {}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_frame_for_an_end_that_is_gone_closes_the_stream_while_a_stranger_s_close_or_an_odd_frame_does_nothing(
|
|
|
|
|
tmp_path, daemon, lineage
|
|
|
|
|
):
|
|
|
|
|
host = _Peer(daemon)
|
|
|
|
|
agent = await _launched(daemon, host, tmp_path)
|
|
|
|
|
bridge = _bridge(daemon, lineage, under=agent)
|
|
|
|
|
stream = (await bridge.ask(protocol.MCP_OPEN, _opening()))["result"]["stream"]
|
|
|
|
|
|
|
|
|
|
await bridge.tell(protocol.MCP_FRAME, {"stream": stream, "text": 5, "end": True})
|
|
|
|
|
await bridge.tell(protocol.MCP_FRAME, {"stream": True, "text": "x", "end": True})
|
|
|
|
|
await _Peer(daemon).tell(protocol.MCP_CLOSE, {"stream": stream, "reason": "not mine to close"})
|
|
|
|
|
assert host.notified(protocol.MCP_FRAME) == [] and stream in daemon.streams
|
|
|
|
|
|
|
|
|
|
host.out.close()
|
|
|
|
|
await bridge.tell(protocol.MCP_FRAME, {"stream": stream, "text": "x", "end": True})
|
|
|
|
|
|
|
|
|
|
assert stream not in daemon.streams
|
|
|
|
|
assert bridge.notified(protocol.MCP_CLOSE) == [
|
|
|
|
|
{"stream": stream, "task": "agent@box-1", "reason": "the other end of the stream is gone"}
|
|
|
|
|
]
|