feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
"""raven.node.client: one live link per registered machine, and the agents it carries.
|
|
|
|
|
|
|
|
|
|
The launch tests need no second computer. A stand-in ``ssh`` on PATH does what
|
|
|
|
|
sshd does with the one remote-command argument it is sent -- hands it to the
|
|
|
|
|
user's shell, in that user's home -- so the real ssh line and the real node
|
|
|
|
|
run: the machine's node directory holds a ``bin/python`` that is this test
|
|
|
|
|
run's own interpreter. Only the network is missing. The stand-in also fails
|
|
|
|
|
the way ssh fails, writing its own address-bearing words to the log the ssh
|
|
|
|
|
line names (``-E``), which is how the tests show those words never reach a
|
|
|
|
|
message.
|
|
|
|
|
|
|
|
|
|
Every failure is one sentence that names the machine and what to do, never the
|
|
|
|
|
address it is reached at.
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
from __future__ import annotations
|
|
|
|
|
|
|
|
|
|
import asyncio
|
|
|
|
|
import hashlib
|
|
|
|
|
import json
|
|
|
|
|
import os
|
feat: enable MCP lending on Windows hosts (#972)
## Summary
Windows hosts can lend granted MCP servers to local and machine-based
ACP agents through authenticated loopback endpoints. Hosts with Unix
sockets retain that transport. Server definitions, upstream credentials,
sandbox policy and disabled-tool enforcement stay on the host.
- Share one connector between the local bridge and node client. Bind TCP
endpoints only to 127.0.0.1 and authenticate with a fresh per-endpoint
token before opening an upstream.
- Read inherited Windows stdin pipes through a daemon worker with
backpressure. Acknowledge each chunk on the event loop before reading
another, so neither the reader buffer nor queued callbacks can grow
without bound.
- Cancel both bridge pumps on exit, allow up to one second for the
endpoint connection to flush, then abort the transport if it still holds
unsent bytes. A peer that stops reading cannot hold failure or
cancellation cleanup indefinitely.
- Resolve this installation's raven.exe, preserve task and session
isolation, hide endpoint credentials from grant representations, and
redact them in bridge diagnostics and session-start journal frames.
Remote dispatches do not receive the host's loopback token.
- Update capability reporting, CLI help, the remote ACP entry's
`machine` docstring, CONTEXT.md and the English/Chinese collaboration
guide. Add Windows CI coverage and exercise both Unix and TCP endpoints
on Linux.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Docker Linux, Debian Bookworm with procps and git, Python 3.12.12,
--init, a native Linux source copy, and a frozen environment: 783
related unit tests and all 3 real-server integration tests passed. The
source and test files were compared with the committed versions before
publication.
```sh
uv sync --frozen --extra dev
uv run --frozen --no-sync pytest tests/test_mcp_transport.py tests/test_mcp_bridge.py tests/test_mcp_endpoint.py tests/test_cli_mcp_commands.py tests/test_subagent_acp_mcp_lifecycle.py tests/test_node_client.py tests/test_acp_journal.py tests/test_subagent_acp_remote.py tests/test_subagent_manager.py tests/test_subagent_mcp_grant.py tests/test_subagent_third_party.py -q -n 4 --dist loadfile --tb=short --cov=raven --cov-report=json:/out/coverage.json --junitxml=/out/units.xml
uv run --frozen --no-sync pytest tests/integration/test_mcp_bridge_e2e.py -m integration -q -n 0 --tb=short --junitxml=/out/integration.xml
```
PR diff coverage is 98.26% (169/172 changed executable lines), above the
unchanged 90% threshold. On the host checkout, `$coverageReport` points
to the JSON exported by that Linux run:
```powershell
uv run --frozen --no-sync python scripts/coverage_gate.py --coverage-json $coverageReport diff --base-ref origin/main --head HEAD --threshold 90
```
Native Windows, Python 3.13.12, a separate frozen environment and
PYTHONUTF8=1: all 21 bridge unit tests passed; all 8 integration cases
passed, comprising 3 MCP bridge cases and 5 native command-line cases
retained in the Windows CI job.
```powershell
uv run --frozen --python 3.13 --no-sync pytest tests/test_mcp_bridge.py -q -n 0 --tb=short
uv run --frozen --python 3.13 --no-sync pytest tests/integration/test_mcp_bridge_e2e.py tests/integration/test_acp_mirasim_real_process.py -m integration -q -n 0 --tb=short
```
Seven regression cases cover paused input, resuming consumption, closing
a paused worker, a busy event loop, and teardown with pending TCP
writes: while the endpoint stays stalled, while it drains those writes,
and when a second cancel arrives after that drain finished. Aborting the
endpoint after a close that finished was wrong: the selector transport
has already run connection_lost and dropped its loop, and its
_force_close returns early only on _conn_lost, which a drained close
does not raise, so the bridge exited 1 at teardown on Linux and macOS.
The abort now depends on whether the endpoint's write buffer still holds
bytes, since a closing transport empties it before calling
connection_lost. A flag set once the close returned would miss the
second cancel, and that case fails with one. The stalled-endpoint
teardown case pauses the endpoint's reading before the frames are fed:
it used to wait for the socket to stop reading through its own flow
control, which took 1.87 s of the case's 2 s budget on Linux, so a busy
runner timed out before the teardown ran. It is now 0.17 s per case, and
20 consecutive runs under 32 busy loops on 16 cores all passed. The
original teardown and buffering cases fail without the fixes. At this
head, making the stdin worker read again without waiting for its
previous chunk to be delivered fails 3 cases, removing the close
deadline fails 2, and never aborting fails 2; aborting unconditionally
fails both drained cases. These mutations ran on a disposable copy of
the source tree, not on the committed files.
The unreachable-upstream integration test still uses its original
1.5-second deadline. Its Windows fixture bypasses the venv redirector
while preserving the venv, so an inherited stdout handle cannot mask
EOF.
Full-PR pre-commit, commitlint, commit-message, source-language, asset
and whitespace checks passed. The bridge type check passed and all 10
import contracts were kept:
```powershell
uv run --frozen --no-sync pre-commit run --from-ref origin/main --to-ref HEAD
npx --no-install commitlint --from origin/main --to HEAD --config commitlint.config.cjs
uv run --frozen --no-sync python scripts/check_commit_messages.py origin/main..HEAD
uv run --frozen --no-sync python scripts/check_large_files.py origin/main...HEAD
uv run --frozen --no-sync python scripts/check_source_language.py origin/main...HEAD
uv run --frozen --no-sync ty check raven/mcp/bridge.py
uv run --frozen --no-sync lint-imports
git diff --check origin/main...HEAD
```
These are focused local results. Remote dispatch tests use the
repository's SSH stand-in and real node processes, not an external
machine. POSIX-dependent paths are covered on Linux; no full-suite
Windows pass, live LLM run or sandbox VM run is claimed. Existing
Windows path, shell and journal portability outside this MCP change
remains out of scope.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
Windows dispatches can now use granted MCP servers. Each fallback
endpoint opens an ephemeral loopback listener with a bounded
authentication handshake and its own random token. Upstream definitions
and credentials remain on the host, and remote dispatches receive no
loopback token.
Input now pauses when a bridge cannot forward requests. During shutdown,
a connection that cannot flush within one second is aborted and its
remaining queued bytes can be discarded. This bounds cleanup when a peer
has stopped reading.
Unix sockets remain the preferred transport, and the Raven node wire
protocol is unchanged. No dependencies, configuration fields or
migrations are added. Reverting the change restores the previous Windows
MCP limitation.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Fixes #962
---------
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-19 19:58:55 +08:00
|
|
|
import secrets
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
import signal
|
|
|
|
|
import sys
|
|
|
|
|
import time
|
|
|
|
|
from pathlib import Path
|
feat(*): lend the host's MCP servers to an acp agent on a machine through its node (#947)
## Summary
Adds MCP lending for remote ACP agents on top of the machine and
recovery
services already merged in #920, #922 and #934. The branch includes
current
main and its default-on registered-machine behavior.
What this does: an acp entry on a registered machine is handed the MCP
servers its row grants, the way an entry on this computer is, and the
servers themselves stay here with their definitions and keys.
- The agent on the machine runs the node's own bridge as a stdio MCP
server (`python -I -m raven.node --mcp --dir <node dir> --task <task>
--lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte
pumps around a connection to the daemon's socket, no MCP parsed, no
credential held, the standard library only.
- The daemon carries the bridge's frames as a *stream* to the host
holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6,
`raven/node/protocol.py`, `raven/node/server.py`). The daemon tells
which
task a bridge is by the process that connected -- the peer pid off the
socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the
parent chain to the agent the daemon started, which the bridge is under
and no other task's process is; the kernel supplies that pid and a peer
cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the
task it names must be the one the connection is a process of. Two tasks
on
one machine run as one user, so no name or file a bridge could present
would tell them apart; the connecting process does. One stream per
bridge, at most 16 per task; a frame
travels in pieces of at most 256 KiB characters, the last marked
(`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the
daemon relays and keeps nothing. A stream ends with either end, the
host's link or the task, and the other end is told.
- The host opens the endpoints for a machine row exactly as for a row
here
(`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in
the host's sandbox, HTTP auth added here), binds them to the machine,
dispatch and acquired agent task
(`NodePool.lend`/`unlend`) before handing out bridge stanzas. On
`mcp/open`
the host checks the daemon-authenticated task against that binding
before
connecting; it rechecks the grant and link after the socket opens. An
unlend, replacement grant or replaced link cannot authorize that pending
connection. The host then pumps the frames both ways
(`raven/node/client.py`).
`node/hello` names the bridge; a node serving without a daemon
(`--stdio`)
names none, and the dispatch then goes without MCP and says so.
- The reply closes with the servers lent and the machine's node, or with
why none were. Remote lending requires Unix-socket support on the host:
on Windows it is withheld before endpoint setup, the task continues, and
the reply explains why. The roster's "injectable MCP" matches that
support
(`session_mcp_delivered`). The cli lane on a machine still
withholds its servers (the next change); the openai lane has no tools.
Not in this change: the cli lane's MCP file on the machine; a stream
that
survives the link going (a bridge sees its server end, the agent reports
the call failed, and the next dispatch works again); MCP for a run
picked
up after this computer restarted (#934): it reports its calls fail.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Follow-up fix 05eaba67a, after the c9de53947 review:
- The nine tests the Windows review named now pin the host fact through
one
fixture, `unix_socket_host` (`tests/conftest.py`), which sets
`raven.mcp.bridge.unix_sockets_available` to true and re-points every
`raven.*` module that imported it by name; their bodies are unchanged.
- Found complete, not just the nine: the whole suite was run with
`asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted
before collection (a pytest plugin, controller and xdist workers), at
the
merge base `6b938a426` and at `c9de53947`, and the two failure sets
compared. 17 node ids failed at the head and not at the base: the nine,
and eight this change added that run the transport for real (the bridge
module, and two lending tests through the stand-in machine). Those eight
are guarded by host capability instead of pinned: the bridge module
skips
when `sys.platform == "win32" or not unix_sockets_available()`, the two
lending tests carry the existing `skipif(not unix_sockets_available())`.
After the fix the same run fails the same 66 node ids at the head as at
the base: nothing this change adds or touches fails on a host without
unix sockets.
- A bridge whose daemon side goes ends as streams end. The host can
close a
stream while a frame of the agent's is on its way; the daemon writes
`mcp/close` and hangs up, and the frame meets the closed socket, or the
read side is reset first. That exited 1 with `BrokenPipeError` or
`ConnectionResetError`; a connection-level end is now exit 0 with
nothing
said, a frame past the cap still exits 1 with its reason, both pumps are
awaited before the socket closes, and a daemon gone before the stream
opens is one sentence, not a traceback. Four deterministic cases with a
stand-in connection fail on `c9de53947` and pass here.
- The bridge test's stand-in agent outlives its bridge, as an agent
outlives
one of its MCP servers, so the later parts of
`test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins`
no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80
here.
- The focused set (node, MCP, remote, manager and roster files) -- 935
passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
simulation cases, which fail the same way on main on this computer);
diff coverage against main 97.67% (420/430). Real machine, node
reinstalled with this tree in a directory of the check's own: a stdlib
ACP
agent's bridge reached a host-held server and 1 MiB and 8 MiB results
crossed whole, bridge exit 0; nothing was left on the machine.
Follow-up fix c9de53947, after the same-user token finding:
- The daemon no longer uses a token. `agent/launch` mints and writes
none,
`mcp.token` and the `hmac` check are gone, and `Daemon.open_stream`
identifies the bridge by the connecting process instead (`_peer_pid` +
`Daemon.task_of`). The same-user path the reviewer kept open -- a peer
reading another task's `mcp.token` and opening its stream -- is closed,
because identity is the connection the kernel reports, not a readable
secret.
- Tests, each failing on the unfixed tree first: a bridge placed under
task
B that names task A is refused (`the connection is a process of task
'agent@box-2', not 'agent@box-1'`), as are a process under no task, an
unplaced connection, and a task the daemon does not run
(`tests/test_node_server.py`); a real-OS test launches an agent, takes
its
real child, and asserts the child resolves to the task while the test's
own
process resolves to none (runs on Linux in CI); `_peer_pid` is checked
against a real socketpair; the in-process and subprocess bridge tests
now
identify by the connecting process, and a bridge this test spawns
itself,
or another task's agent spawns, never opens the stream.
- Windows withholding now covers a row on this computer too, not only a
machine row: `session_mcp_delivered` drops the `machine and` guard and
answers no whenever `unix_sockets_available()` is false, so a local acp
row is no longer advertised as injectable and no longer raises at
dispatch
on a host without the transport; `_mcp_endpoints` yields nothing before
opening an endpoint and `_mcp_note` explains it. The two tests the
Windows
review named now guard on `unix_sockets_available()`, and the
withholding
test is parameterized over a machine row and a local row.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over the node/MCP/remote suite -- 97.63% changed-line coverage
(412/422).
`ruff`, `ty` on the changed modules, and the import-cycle budget pass.
- Real machine (ravenx2, this head, node reinstalled, digest
fdbe132e76f0):
a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB
and
an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the
connecting-process identity admits the legitimate bridge end to end on
real Linux; the cross-task refusal on Linux is the real-OS test above,
which CI runs there.
Earlier review fixes (4dc5b3e1f, the task-to-grant binding). Its
host-side
binding of each dispatch's endpoints to the acquired agent task is kept;
the
daemon-side token this run verified was since replaced by the
connecting-process
identity above (c9de53947), so the token-named tests and reproduction
below no
longer describe the current tree.
- `uv run --no-sync pytest tests/test_node_client.py
tests/test_node_server.py tests/test_node_bridge.py
tests/test_node_relay.py tests/test_node_bundle.py
tests/test_node_install.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py
tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict
--cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report=
--tb=short` -- 1495 passed and one minimal-node stdio response test
failed;
strict idle accounting also flagged the assembly test. Both cases pass
together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or
timing limits were weakened. The new authorization, revocation and host
compatibility tests all pass.
- The five focused cases failed on the unfixed implementation first: a
different authenticated task selecting another grant, a stream with no
task, each missing Unix socket API, and an astral Unicode chunk over the
node's byte limit. Regressions also cover revocation, grant replacement
and link replacement while an endpoint connection is opening, and a real
remote turn continuing when both host Unix socket APIs are absent.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 97.94% (381/389 changed executable lines); this fix alone is 100%
(40/40). Python lint, type, dependency
and import checks pass; all 10 import contracts hold. The scope checks
for commit messages, source language and file sizes pass. Both RPC
generators pass `--check` after integrating current main.
- A real Linux node with two concurrent tasks and a host loopback
endpoint:
before, the attacker's own task token plus the victim's public grant
names returns the victim endpoint's marker; after, it returns no data
and the endpoint is never reached. Its own grant works, unknown and
revoked grants fail, and the same attack remains denied after the relay
is killed and an ordinary ping reattaches both tasks. A full chunk of
262144 astral characters was lost/corrupted before and arrives intact
after. The final documented source tree was reinstalled and checked
again.
- Real ACP dispatch through the node's bridge reaches a host-held stdio
MCP tool and returns its marker; its grant is removed when the turn
ends.
With the host Unix socket APIs removed to simulate Windows, the remote
agent still completes, receives no MCP, explains the host limitation,
and the host MCP ledger records no extra call. This is not a native
Windows run. Dedicated test tasks, host endpoints and bridge processes
are cleaned up.
- CONTEXT.md, module/field docstrings, both collaboration guides, the
external design notes and this description were checked against the
implementation. Main's no-debug-switch behavior and full-turn recovery
documentation are retained.
Full combined-tree verification: `uv run --no-sync pytest -n 4
--tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure
reproduces on current main in a separate checkout: host CLI color
rendering,
LibreOffice detection and the two simulation cases. No other test fails.
The minimal-node response test that failed in the focused parallel
selection
passes in this full run. Diff coverage: 97.94% (381/389 executable
changed lines).
Earlier author verification:
- `uv run pytest tests/test_node_bridge.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_bundle.py tests/test_node_install.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py`
-- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge
and a stdio node naming none, the token written owner-only beside the
record, a stream opened with the task's token and refused with another
or for a task not running, refused with no host holding the task,
refused past 16 per task, one per bridge connection, the pieces of a
frame crossing both ways as they are and a stranger carrying nothing,
and every way a stream ends (the bridge closing, the host closing, the
bridge's connection ending, the host's link ending, the agent ending)
with the other end told; the bridge as a process against a real daemon
over its socket: a frame to the host and one back whole, a large frame
each way in pieces, the agent closing its stdin ending the stream and
the bridge leaving clean, the host closing the stream ending the bridge,
a wrong token and a missing daemon each refused on stderr with exit 1;
the stand-in machine with the real daemon and the real bridge: an acp
stub there that connects to its `mcpServers` and calls a tool reaches a
stdio server here and answers with its result, the stanza handed to it
naming the node's bridge, the task, the dispatch and the server and
nothing of the server's definition or socket, the ledger here showing
one call, no bridge outliving the dispatch; the reply's note and the
roster predicate for a machine row.
- `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed,
4 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`,
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`
and
`test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`,
which fail
the same way on main on this machine.
- Real machine (a registered Linux machine with Claude Code signed in,
the node reinstalled with the code before and after):
A28, before (the
code of #934's head and its node): the turn closed with "MCP servers
'probe' were not lent", Claude Code said no tool of that name was
connected, and the server here saw no call; after: `PROBE_OK:r2-lend`
in 9.4 s, the server here recorded the call, and the turn closed with
"lent to acp agent ... through the Raven node on machine ...". A34:
after
the turn the machine's process list shows no bridge, a grep of its node
directory finds nothing of the server's command or of this computer's
socket directory, and no endpoint socket is left here. A29: with the
tool switched off here, Claude Code answered UNAVAILABLE and the server
saw no call. A30: a 40 s tool call under way, the bridge visible in the
machine's process list, then cancelled: 3 s later no bridge there and
no socket here. A31: the relay ssh killed during a 20 s call: the turn
ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next
dispatch reached the server again in 7.6 s. A32: five calls in one turn
all reached the server and came back; an 8 MiB answer: the server here
produced it, and a plain ACP agent on the machine (a stdlib stub that
speaks the stanza's bridge directly) received it whole through the node
-- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude
Code itself closed the server on that answer and reported the call
failed: the agent's own cap on a tool result, not the link's.
- `ruff`, the commit-message and PR-body checks -- clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese collaboration guides describe lending,
task-bound authorization,
Windows withholding, and the existing CLI/restart limitations.
## Risk
Registered-machine support is enabled by configuration, without the
removed
trial environment switch. An ACP entry on a machine receives its granted
MCP servers on a Unix-socket-capable host; the host's normal MCP policy
still
applies. This intentionally exposes those tools to the selected remote
agent.
The server definition, credentials and endpoint paths remain on the
host.
The daemon identifies a bridge's task by the connecting process (the
peer
pid the kernel reports, walked to the agent it started), not by any
token or
name the bridge presents, so a peer task sharing the OS account cannot
open
another task's stream; the host then authorizes that task against the
dispatch/server grant. A system that reports no peer pid opens no
stream.
On Windows hosts remote lending is withheld explicitly, without aborting
the
remote task. Protocol 6 remains the MCP wire version; host and node must
also
have the same code digest, so updated nodes must be reinstalled.
Rollback is
a revert; the machine then carries no stream and receives no bridge
stanza.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#934 (merged); #920 and #922 (merged).
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-15 15:37:50 +08:00
|
|
|
from types import SimpleNamespace
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
from typing import Any
|
|
|
|
|
|
|
|
|
|
import pytest
|
|
|
|
|
|
|
|
|
|
from raven.acp_client import remote
|
|
|
|
|
from raven.acp_client.client import AcpClient
|
|
|
|
|
from raven.acp_client.protocol import AcpConnectionError, AcpRemoteError, AcpTimeoutError
|
|
|
|
|
from raven.agent.subagent.backends import env as backend_env
|
|
|
|
|
from raven.agent.tools.filesystem import ReadFileTool
|
|
|
|
|
from raven.node import bundle, protocol
|
|
|
|
|
from raven.node import client as node_client
|
|
|
|
|
from raven.node.client import DEFAULT_NODE_DIR, ChannelEndpoint, NodeError, NodePool, _Node
|
|
|
|
|
|
|
|
|
|
HOST = "203.0.113.7"
|
|
|
|
|
PORT = 58717
|
|
|
|
|
|
|
|
|
|
_FAKE_SSH = """#!{python}
|
|
|
|
|
import json, os, sys
|
|
|
|
|
here = os.path.dirname(os.path.abspath(__file__))
|
|
|
|
|
args = sys.argv[1:]
|
|
|
|
|
with open(os.path.join(here, "calls"), "a") as f:
|
|
|
|
|
f.write(json.dumps(args) + "\\n")
|
|
|
|
|
mode = open(os.path.join(here, "mode")).read().strip()
|
|
|
|
|
home = open(os.path.join(here, "home")).read().strip()
|
|
|
|
|
if mode == "unreachable":
|
|
|
|
|
said = "ssh: connect to host {host} port {port}: Operation timed out"
|
|
|
|
|
log = args[args.index("-E") + 1] if "-E" in args else None
|
|
|
|
|
if log:
|
|
|
|
|
open(log, "a").write(said + "\\n")
|
|
|
|
|
else:
|
|
|
|
|
sys.stderr.write(said + "\\n")
|
|
|
|
|
sys.exit(255)
|
feat(*): a machines hub tab, and registering a machine with a password Raven keeps (#1005)
## Summary
Add a Machines tab to the Agent Hub and let owners register an SSH
machine with either a key or a password.
- The Machines tab reuses the existing machine cards, details and
registration form. The hub remembers the last selected module. Machine
details show the login method, and removal explains that the locally
stored password is deleted.
- Password registrations keep a 0600 credential file under the Raven
home's 0700 keys directory. Registration, node installation, file tools,
tunnels and Oncall use the shared SSH askpass transport. Passwords are
excluded from command lines, registry rows and responses, and included
in held-secret scrubbing.
- Password connections authenticate independently, disable inherited
jump hosts, and answer only the registered account's password prompts.
Oncall cannot redirect a stored password to another endpoint. Windows
rejects this mode before creating credentials because its OpenSSH cannot
execute Raven's POSIX helper. Key authentication remains supported as
before.
- Concurrent registrations preserve the successful request's password.
Each probe uses its own temporary 0600 credential file, which is
scrubbed and removed afterwards. Password writes receive a file
identity; cleanup checks ownership under the writer's lock. Successful
registration checks duplicates again and publishes its password with the
registry row under that lock. Both completion orders, RPC cancellation
and CLI aborts are covered.
The RPC contract adds optional `MachineRow.auth` and password
registration parameters. English and Chinese machine documentation, the
runtime context and changelog describe the resulting behavior.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
After rebasing onto the latest main:
```bash
uv run --no-sync pytest tests/test_ops_connection_add.py tests/test_ops_connections.py tests/test_ops_transport.py tests/test_cli_ops_connection_commands.py tests/test_rpc_machines.py tests/test_held_secrets_machine_passwords.py tests/test_acp_remote.py tests/test_node_client.py tests/test_rpc_schema_match.py tests/test_rpc_registration.py tests/test_import_cycle_budget.py -q -n 0 --tb=short --cov=raven.ops.connection_add --cov=raven.ops.transport --cov=raven.cli.ops_connection_commands --cov=raven.rpc.methods.machines --cov-report=json:/tmp/pr1005-fix-coverage.json
```
Result: 932 passed, 6 skipped. The skipped cases require zsh, tcsh or
csh, which are absent on this Linux host.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
/tmp/pr1005-fix-coverage.json diff --base-ref HEAD^ --threshold 99`:
100% (94/94 executable changed lines) for the concurrency fix.
- `uv run --no-sync pre-commit run --from-ref HEAD^ --to-ref HEAD`:
passed, including formatting, file checks and private-key detection.
- `uv run --no-sync ty check raven/ops/connection_add.py
raven/ops/transport.py raven/cli/ops_connection_commands.py
raven/rpc/methods/machines.py`: passed.
- `uv run --no-sync lint-imports`: all 10 contracts passed.
- `uv run --no-sync python scripts/check_source_language.py
origin/main..HEAD`, `uv run --no-sync python
scripts/check_commit_messages.py origin/main..HEAD`, `make
check-large-files`, and `git diff --check origin/main...HEAD`: passed.
The original RPC failure/cancellation and CLI-abort reproductions failed
before the fix. Additional regressions cover the reverse completion
order, probes with competing passwords, temporary-secret scrubbing,
duplicate registration and failed registry writes. The complete rebased
source was compared with the original PR plus main, and the concurrency
fix retained its patch identity.
## Risk
- [x] Security impact considered: stored and temporary passwords are
owner-readable and scrubbed from tool output. Unknown SSH host keys are
accepted on the first connection, as before; the guides explain
verifying the host from a terminal first on an untrusted network.
Passwords shorter than six characters or containing CR/LF are rejected.
- [x] Backward compatibility considered: key registration remains the
default and `MachineRow.auth` is optional. Password publication and
cleanup now use file locks and atomic replacement; temporary probe files
are removed when the probe ends.
- [x] Rollback path is clear: revert this PR to restore the earlier
registration flow. Removing a machine deletes the password stored under
its id. Existing password rows require migration to key authentication
if reverting the feature.
## Related Issues
N/A
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn>
Co-authored-by: MiniMax (minimax-m3.1-flash-free) <noreply@minimax.io>
Co-authored-by: Codex <noreply@openai.com>
2026-10-20 12:23:40 +08:00
|
|
|
expect = os.path.join(here, "password")
|
|
|
|
|
if os.path.exists(expect):
|
|
|
|
|
# The machine takes a password: the way OpenSSH asks for one under SSH_ASKPASS_REQUIRE=force.
|
|
|
|
|
import subprocess as sp
|
|
|
|
|
log = args[args.index("-E") + 1] if "-E" in args else None
|
|
|
|
|
def refuse(said):
|
|
|
|
|
if log:
|
|
|
|
|
open(log, "a").write(said + "\\n")
|
|
|
|
|
else:
|
|
|
|
|
sys.stderr.write(said + "\\n")
|
|
|
|
|
sys.exit(255)
|
|
|
|
|
if "-i" in args or os.environ.get("SSH_ASKPASS_REQUIRE") != "force" or not os.environ.get("SSH_ASKPASS"):
|
|
|
|
|
refuse("worker@{host}: Permission denied (password).")
|
|
|
|
|
prompt = os.environ.get("RAVEN_MACHINE_PASSWORD_FOR", "") + "'s password: "
|
|
|
|
|
given = sp.run([os.environ["SSH_ASKPASS"], prompt], capture_output=True, text=True).stdout.rstrip("\\n")
|
|
|
|
|
if given != open(expect).read().strip():
|
|
|
|
|
refuse("worker@{host}: Permission denied (password).")
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
command = args[args.index("--") + 1] if "--" in args else args[-1]
|
|
|
|
|
os.chdir(home)
|
|
|
|
|
# A daemon these tests start leaves soon after they do; the relay reads the idle time from its environment.
|
|
|
|
|
os.execve("/bin/sh", ["/bin/sh", "-c", command], {{"HOME": home, "SHELL": "/bin/sh", "PATH": os.environ["PATH"], "RAVEN_NODE_IDLE_S": "2"}})
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
# A node that misbehaves in one named way, for the handshake's refusals.
|
|
|
|
|
_STUB_NODE = """
|
|
|
|
|
import json, sys
|
|
|
|
|
mode, digest, proto = sys.argv[1], sys.argv[2], int(sys.argv[3])
|
|
|
|
|
if mode == "crash":
|
|
|
|
|
sys.stderr.write("Traceback (most recent call last):\\n File x\\nModuleNotFoundError: No module named 'loguru'\\n")
|
|
|
|
|
sys.exit(1)
|
|
|
|
|
for line in sys.stdin:
|
|
|
|
|
line = line.strip()
|
|
|
|
|
if not line:
|
|
|
|
|
continue
|
|
|
|
|
request = json.loads(line)
|
|
|
|
|
if mode == "silent" or (mode == "hello-only" and request["method"] != "node/hello"):
|
|
|
|
|
continue
|
|
|
|
|
answer = {"jsonrpc": "2.0", "id": request["id"]}
|
|
|
|
|
if mode == "refuse":
|
|
|
|
|
answer["error"] = {"code": -32600, "message": "this node speaks protocol 1, not 2"}
|
|
|
|
|
elif mode == "other-protocol":
|
|
|
|
|
answer["result"] = {"protocol": proto + 1, "digest": digest}
|
|
|
|
|
elif mode == "other-code":
|
|
|
|
|
answer["result"] = {"protocol": proto, "digest": "000000000000"}
|
|
|
|
|
elif mode == "not-an-object":
|
|
|
|
|
answer["result"] = ["node/hello"]
|
|
|
|
|
else:
|
|
|
|
|
answer["result"] = {"protocol": proto, "digest": digest, "python": "3.12.0", "home": "", "rg": False}
|
|
|
|
|
print(json.dumps(answer), flush=True)
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
# An agent the node starts for the tests: answers every request with its
|
|
|
|
|
# params, says one thing on stderr, leaves on `quit` with the code asked for,
|
|
|
|
|
# and on `spawn` starts a child that outlives it unless its group is ended.
|
|
|
|
|
_ECHO_AGENT = """
|
|
|
|
|
import json, subprocess, sys
|
|
|
|
|
sys.stderr.write("agent: started\\n")
|
|
|
|
|
sys.stderr.flush()
|
|
|
|
|
for line in sys.stdin:
|
|
|
|
|
line = line.strip()
|
|
|
|
|
if not line:
|
|
|
|
|
continue
|
|
|
|
|
request = json.loads(line)
|
|
|
|
|
method = request.get("method")
|
|
|
|
|
if method == "quit":
|
|
|
|
|
sys.exit(int(request["params"]["rc"]))
|
|
|
|
|
if method == "spawn":
|
|
|
|
|
child = subprocess.Popen([sys.executable, "-c", "import time; time.sleep(120)"])
|
|
|
|
|
print(json.dumps({"jsonrpc": "2.0", "id": request["id"], "result": {"pid": child.pid}}), flush=True)
|
|
|
|
|
continue
|
|
|
|
|
print(json.dumps({"jsonrpc": "2.0", "id": request["id"], "result": {"echo": request.get("params")}}), flush=True)
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class Machine:
|
|
|
|
|
"""The stand-in machine: its home, its ssh, and what was asked of it."""
|
|
|
|
|
|
|
|
|
|
def __init__(self, root: Path, monkeypatch: pytest.MonkeyPatch, rows: list[dict[str, Any]] | None = None) -> None:
|
|
|
|
|
self.root = root
|
|
|
|
|
self.home = root / "box-home"
|
|
|
|
|
self.home.mkdir()
|
|
|
|
|
self.bin = root / "fakebin"
|
|
|
|
|
self.bin.mkdir()
|
|
|
|
|
ssh = self.bin / "ssh"
|
|
|
|
|
ssh.write_text(_FAKE_SSH.format(python=sys.executable, host=HOST, port=PORT), encoding="utf-8")
|
|
|
|
|
ssh.chmod(0o755)
|
|
|
|
|
(self.bin / "home").write_text(str(self.home), encoding="utf-8")
|
|
|
|
|
self.mode("ok")
|
|
|
|
|
self.registry = root / "connections.json"
|
|
|
|
|
self.rows(rows or [row()])
|
|
|
|
|
monkeypatch.setenv("RAVEN_CONNECTIONS", str(self.registry))
|
|
|
|
|
path = f"{self.bin}{os.pathsep}{os.environ.get('PATH', '/usr/bin:/bin')}"
|
|
|
|
|
monkeypatch.setattr(backend_env, "_LOGIN_ENV", {**os.environ, "PATH": path})
|
|
|
|
|
monkeypatch.setattr("raven.config.paths.get_logs_dir", lambda: root / "logs")
|
|
|
|
|
|
|
|
|
|
def rows(self, rows: list[dict[str, Any]]) -> None:
|
|
|
|
|
self.registry.write_text(json.dumps({"connections": rows}), encoding="utf-8")
|
|
|
|
|
|
|
|
|
|
def mode(self, mode: str) -> None:
|
|
|
|
|
(self.bin / "mode").write_text(mode, encoding="utf-8")
|
|
|
|
|
|
feat(*): a machines hub tab, and registering a machine with a password Raven keeps (#1005)
## Summary
Add a Machines tab to the Agent Hub and let owners register an SSH
machine with either a key or a password.
- The Machines tab reuses the existing machine cards, details and
registration form. The hub remembers the last selected module. Machine
details show the login method, and removal explains that the locally
stored password is deleted.
- Password registrations keep a 0600 credential file under the Raven
home's 0700 keys directory. Registration, node installation, file tools,
tunnels and Oncall use the shared SSH askpass transport. Passwords are
excluded from command lines, registry rows and responses, and included
in held-secret scrubbing.
- Password connections authenticate independently, disable inherited
jump hosts, and answer only the registered account's password prompts.
Oncall cannot redirect a stored password to another endpoint. Windows
rejects this mode before creating credentials because its OpenSSH cannot
execute Raven's POSIX helper. Key authentication remains supported as
before.
- Concurrent registrations preserve the successful request's password.
Each probe uses its own temporary 0600 credential file, which is
scrubbed and removed afterwards. Password writes receive a file
identity; cleanup checks ownership under the writer's lock. Successful
registration checks duplicates again and publishes its password with the
registry row under that lock. Both completion orders, RPC cancellation
and CLI aborts are covered.
The RPC contract adds optional `MachineRow.auth` and password
registration parameters. English and Chinese machine documentation, the
runtime context and changelog describe the resulting behavior.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
After rebasing onto the latest main:
```bash
uv run --no-sync pytest tests/test_ops_connection_add.py tests/test_ops_connections.py tests/test_ops_transport.py tests/test_cli_ops_connection_commands.py tests/test_rpc_machines.py tests/test_held_secrets_machine_passwords.py tests/test_acp_remote.py tests/test_node_client.py tests/test_rpc_schema_match.py tests/test_rpc_registration.py tests/test_import_cycle_budget.py -q -n 0 --tb=short --cov=raven.ops.connection_add --cov=raven.ops.transport --cov=raven.cli.ops_connection_commands --cov=raven.rpc.methods.machines --cov-report=json:/tmp/pr1005-fix-coverage.json
```
Result: 932 passed, 6 skipped. The skipped cases require zsh, tcsh or
csh, which are absent on this Linux host.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
/tmp/pr1005-fix-coverage.json diff --base-ref HEAD^ --threshold 99`:
100% (94/94 executable changed lines) for the concurrency fix.
- `uv run --no-sync pre-commit run --from-ref HEAD^ --to-ref HEAD`:
passed, including formatting, file checks and private-key detection.
- `uv run --no-sync ty check raven/ops/connection_add.py
raven/ops/transport.py raven/cli/ops_connection_commands.py
raven/rpc/methods/machines.py`: passed.
- `uv run --no-sync lint-imports`: all 10 contracts passed.
- `uv run --no-sync python scripts/check_source_language.py
origin/main..HEAD`, `uv run --no-sync python
scripts/check_commit_messages.py origin/main..HEAD`, `make
check-large-files`, and `git diff --check origin/main...HEAD`: passed.
The original RPC failure/cancellation and CLI-abort reproductions failed
before the fix. Additional regressions cover the reverse completion
order, probes with competing passwords, temporary-secret scrubbing,
duplicate registration and failed registry writes. The complete rebased
source was compared with the original PR plus main, and the concurrency
fix retained its patch identity.
## Risk
- [x] Security impact considered: stored and temporary passwords are
owner-readable and scrubbed from tool output. Unknown SSH host keys are
accepted on the first connection, as before; the guides explain
verifying the host from a terminal first on an untrusted network.
Passwords shorter than six characters or containing CR/LF are rejected.
- [x] Backward compatibility considered: key registration remains the
default and `MachineRow.auth` is optional. Password publication and
cleanup now use file locks and atomic replacement; temporary probe files
are removed when the probe ends.
- [x] Rollback path is clear: revert this PR to restore the earlier
registration flow. Removing a machine deletes the password stored under
its id. Existing password rows require migration to key authentication
if reverting the feature.
## Related Issues
N/A
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn>
Co-authored-by: MiniMax (minimax-m3.1-flash-free) <noreply@minimax.io>
Co-authored-by: Codex <noreply@openai.com>
2026-10-20 12:23:40 +08:00
|
|
|
def password(self, expected: str) -> None:
|
|
|
|
|
"""The machine takes a password from now on, and only this one."""
|
|
|
|
|
(self.bin / "password").write_text(expected, encoding="utf-8")
|
|
|
|
|
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
def node(self, where: Path | None = None, *, stub: str | None = None) -> Path:
|
|
|
|
|
"""Put a node of this Raven's code under ``where`` (the default node dir)."""
|
|
|
|
|
base = where or self.home / DEFAULT_NODE_DIR.removeprefix("~/")
|
|
|
|
|
python = base / bundle.install_name() / "bin" / "python"
|
|
|
|
|
python.parent.mkdir(parents=True)
|
|
|
|
|
if stub is None:
|
|
|
|
|
body = f'exec {sys.executable} "$@"\n'
|
|
|
|
|
else:
|
|
|
|
|
script = self.root / "stub_node.py"
|
|
|
|
|
script.write_text(_STUB_NODE, encoding="utf-8")
|
|
|
|
|
body = f"exec {sys.executable} {script} {stub} {bundle.digest()} {protocol.PROTOCOL}\n"
|
|
|
|
|
python.write_text("#!/bin/sh\n" + body, encoding="utf-8")
|
|
|
|
|
python.chmod(0o755)
|
|
|
|
|
return python
|
|
|
|
|
|
|
|
|
|
def agent(self) -> str:
|
|
|
|
|
"""The command of an agent on this machine: the echo agent above."""
|
|
|
|
|
script = self.root / "echo_agent.py"
|
|
|
|
|
script.write_text(_ECHO_AGENT, encoding="utf-8")
|
|
|
|
|
return f"{sys.executable} {script}"
|
|
|
|
|
|
|
|
|
|
def launches(self) -> list[list[str]]:
|
|
|
|
|
path = self.bin / "calls"
|
|
|
|
|
if not path.exists():
|
|
|
|
|
return []
|
|
|
|
|
calls = [json.loads(line) for line in path.read_text(encoding="utf-8").splitlines()]
|
|
|
|
|
return [argv for argv in calls if "-T" in argv]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def row(**extra: Any) -> dict[str, Any]:
|
|
|
|
|
return {
|
|
|
|
|
"id": "box",
|
|
|
|
|
"display_name": "Lab box",
|
|
|
|
|
"host": HOST,
|
|
|
|
|
"port": PORT,
|
|
|
|
|
"user": "worker",
|
|
|
|
|
"key": "/keys/id_test",
|
|
|
|
|
**extra,
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture
|
|
|
|
|
def box(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Machine:
|
|
|
|
|
if not Path("/bin/sh").exists():
|
|
|
|
|
pytest.skip("the stand-in ssh hands the ssh line to a POSIX shell")
|
|
|
|
|
return Machine(tmp_path, monkeypatch)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture
|
|
|
|
|
async def pool():
|
|
|
|
|
nodes = NodePool()
|
|
|
|
|
yield nodes
|
|
|
|
|
await nodes.close_all()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _no_address(message: str) -> None:
|
|
|
|
|
assert HOST not in message and str(PORT) not in message and "worker@" not in message
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def _until(check: Any, *, timeout: float = 10.0) -> None:
|
|
|
|
|
deadline = time.monotonic() + timeout
|
|
|
|
|
while not check():
|
|
|
|
|
assert time.monotonic() < deadline, "did not happen in time"
|
|
|
|
|
await asyncio.sleep(0.02)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _gone(pid: int) -> bool:
|
|
|
|
|
try:
|
|
|
|
|
os.kill(pid, 0)
|
|
|
|
|
except ProcessLookupError:
|
|
|
|
|
return True
|
|
|
|
|
try:
|
|
|
|
|
# Reaped by nobody yet: a zombie answers the signal but is gone.
|
|
|
|
|
done, _ = os.waitpid(pid, os.WNOHANG)
|
|
|
|
|
return done == pid
|
|
|
|
|
except ChildProcessError:
|
|
|
|
|
return False
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- a node over the real ssh line ------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_one_node_answers_every_call_to_its_machine_as_the_tool_would_here(box, pool):
|
|
|
|
|
box.node()
|
|
|
|
|
work = box.home / "work"
|
|
|
|
|
work.mkdir()
|
|
|
|
|
(work / "notes.txt").write_text("".join(f"note {i}\n" for i in range(1, 41)), encoding="utf-8")
|
|
|
|
|
params = {"path": str(work / "notes.txt"), "offset": 10, "limit": 3, "roots": [str(work)]}
|
|
|
|
|
|
|
|
|
|
first = await pool.call("box", protocol.READ, params)
|
|
|
|
|
second = await pool.call("box", protocol.READ, params)
|
|
|
|
|
|
|
|
|
|
local = ReadFileTool(workspace=work, allowed_dirs=(work,), follow_binding=False)
|
|
|
|
|
assert first == second
|
|
|
|
|
assert first["text"] == await local.execute(str(work / "notes.txt"), offset=10, limit=3)
|
|
|
|
|
assert first["sha256"] == hashlib.sha256((work / "notes.txt").read_bytes()).hexdigest()
|
|
|
|
|
assert len(box.launches()) == 1, "one ssh serves every call"
|
|
|
|
|
assert pool.live() == ["box"]
|
|
|
|
|
hello = await pool.hello("box")
|
|
|
|
|
assert hello["digest"] == bundle.digest() and hello["home"] == str(box.home)
|
|
|
|
|
assert hello["protocol"] == protocol.PROTOCOL and hello["limits"]["agents"] == protocol.MAX_AGENTS
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_the_node_is_started_from_where_the_registry_keeps_nodes_isolated_and_without_a_login_shell(
|
|
|
|
|
box, pool, tmp_path
|
|
|
|
|
):
|
|
|
|
|
kept = tmp_path / "nodes on a big disk"
|
|
|
|
|
box.rows([row(node_dir=f"{kept}/")])
|
|
|
|
|
box.node(kept)
|
|
|
|
|
|
|
|
|
|
hello = await pool.hello("box")
|
|
|
|
|
|
|
|
|
|
assert hello["digest"] == bundle.digest()
|
|
|
|
|
(launch,) = box.launches()
|
|
|
|
|
command = launch[launch.index("--") + 1]
|
|
|
|
|
assert str(kept / bundle.install_name() / "bin" / "python") in command
|
|
|
|
|
# Isolated: the login shell's PYTHONPATH or PYTHONHOME must not reach the
|
|
|
|
|
# node, where a shadow package would answer with another digest.
|
|
|
|
|
assert " -I -m raven.node --attach --dir " in command
|
|
|
|
|
# One plain line any login shell reads, from the directory the login
|
|
|
|
|
# lands in: no profile, no directory made, nothing read from stdin ahead
|
|
|
|
|
# of the protocol. The node captures the login environment itself.
|
|
|
|
|
assert command.startswith("exec /bin/sh -c 'echo; ")
|
|
|
|
|
assert "mkdir" not in command and "-lic" not in command and "read -r" not in command
|
|
|
|
|
assert "-T" in launch and "-E" in launch and "ServerAliveInterval=15" in launch
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_node_that_ended_is_started_again_on_the_next_call(box, pool):
|
|
|
|
|
box.node()
|
|
|
|
|
await pool.hello("box")
|
|
|
|
|
await pool._nodes["box"].client.close()
|
|
|
|
|
assert pool.live() == []
|
|
|
|
|
|
|
|
|
|
await pool.hello("box")
|
|
|
|
|
|
|
|
|
|
assert len(box.launches()) == 2
|
|
|
|
|
assert pool.live() == ["box"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_machine_without_the_node_says_how_to_install_it(box, pool):
|
|
|
|
|
with pytest.raises(NodeError) as caught:
|
|
|
|
|
await pool.hello("box")
|
|
|
|
|
|
|
|
|
|
assert str(caught.value) == (
|
|
|
|
|
f"Raven node {bundle.install_name()} is not installed on machine 'Lab box' (box); "
|
|
|
|
|
"install it with `raven ops connection install-node box`"
|
|
|
|
|
)
|
|
|
|
|
assert pool.live() == []
|
|
|
|
|
assert pool.status("box").state == "error" and "is not installed" in pool.status("box").detail
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_an_unreachable_machine_is_named_without_its_address(box, pool):
|
|
|
|
|
box.node()
|
|
|
|
|
box.mode("unreachable")
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError) as caught:
|
|
|
|
|
await pool.hello("box")
|
|
|
|
|
|
|
|
|
|
assert str(caught.value) == "machine 'Lab box' (box) did not answer within the connection timeout"
|
|
|
|
|
_no_address(str(caught.value))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
@pytest.mark.parametrize("stub", ["other-code", "other-protocol", "not-an-object"])
|
|
|
|
|
async def test_a_node_of_other_code_is_refused_with_the_command_that_updates_it(box, pool, stub):
|
|
|
|
|
box.node(stub=stub)
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError) as caught:
|
|
|
|
|
await pool.hello("box")
|
|
|
|
|
|
|
|
|
|
assert str(caught.value) == (
|
|
|
|
|
"the Raven node on machine 'Lab box' (box) runs different code from this Raven; "
|
|
|
|
|
"update it with `raven ops connection install-node box`"
|
|
|
|
|
)
|
|
|
|
|
assert pool.live() == []
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_node_that_refuses_the_handshake_says_why(box, pool):
|
|
|
|
|
box.node(stub="refuse")
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError) as caught:
|
|
|
|
|
await pool.hello("box")
|
|
|
|
|
|
|
|
|
|
assert str(caught.value) == (
|
|
|
|
|
"the Raven node on machine 'Lab box' (box) did not complete its handshake: this node speaks protocol 1, not 2"
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_node_that_cannot_start_gives_its_own_last_word(box, pool):
|
|
|
|
|
box.node(stub="crash")
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError) as caught:
|
|
|
|
|
await pool.hello("box")
|
|
|
|
|
|
|
|
|
|
assert str(caught.value) == (
|
|
|
|
|
"the Raven node on machine 'Lab box' (box) stopped: ModuleNotFoundError: No module named 'loguru'"
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_node_that_never_answers_the_handshake_is_given_up_on(box, pool, monkeypatch):
|
|
|
|
|
monkeypatch.setattr(node_client, "_HELLO_TIMEOUT_S", 0.5)
|
|
|
|
|
box.node(stub="silent")
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError, match=r"^the Raven node on machine 'Lab box' \(box\) did not complete its handshake"):
|
|
|
|
|
await pool.hello("box")
|
|
|
|
|
|
|
|
|
|
assert pool.live() == []
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_call_that_runs_out_of_time_closes_the_node(box, pool):
|
|
|
|
|
box.node(stub="hello-only")
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError) as caught:
|
|
|
|
|
await pool.call("box", protocol.READ, {"path": "x", "roots": ["/"]}, timeout=1.0)
|
|
|
|
|
|
|
|
|
|
assert str(caught.value) == "the Raven node on machine 'Lab box' (box) did not answer fs/read within 1s"
|
|
|
|
|
assert pool.live() == [] and "box" not in pool._nodes
|
|
|
|
|
assert pool.status("box").state == "error"
|
|
|
|
|
|
|
|
|
|
|
feat(*): pick up a turn left on a registered machine when this computer stops (#934)
## Summary
Registered machines are now available without a debug environment
switch.
`RAVEN_DEBUG_MACHINES` previously defaulted to off: it kept
machine-backed
agents out of the dispatch table, refused node/installer access, hid
remote
preset destinations, and treated file paths locally while off. It was a
trial feature gate, not a verbose-logging option. At the owner's
request,
this PR removes the variable and all of those checks. Normal startup and
restart recovery use the configured machine entries; even a leftover
value
of `0` is ignored. Machine registration, SSH authentication, file
permission
prompts, and concurrency limits still apply.
Fourth change in the series that lets a Raven dispatch work to the
machines in its machine registry. #920, #921 (merged through #922) and
#922 are on main; this change stands on main alone now, and the diff is
its own.
What this does: a turn running on a registered machine survives this
computer stopping, and the next start picks it up and reports it.
- While a turn runs on a machine, this computer remembers what the
machine does not know (`raven/acp_client/remote_runs.py`): one
owner-only file per run under `<raven home>/machines/runs/`, with the
machine's registry id, the task id its daemon runs the agent under, the
agent, the session key, the instance handle, the dispatch id, the ACP
session, the id of the prompt whose answer is awaited, the ids this
side has used, how many of the agent's frames were read here, the
agent's answer to `initialize` and the connection's launch key. Kept
current after every frame the connection has dealt with, removed when
the turn ends here. No address is in it.
- When this process stops -- the gateway, `serve`, the TUI, the page
stack -- the
agents on machines stay with their daemons
(`remote.leave_agents_running()` before the drain): a cancelled turn
on a machine sends no `session/cancel`, closing its connection ends the
endpoint only, and the manager leaves the dispatch's record open rather
than finishing it as cancelled. A hard kill or `/restart` leaves the
same record, written as the turn ran.
- At the next start -- the gateway once its generation is bound; the
`serve`, TUI and page stacks once --
`raven/agent/subagent/resume_remote.py`
picks every run up: `task/list` on the machine, `agent/attach` from the
frame last read (`NodePool.attach`), the connection adopted into the
pool with request ids carried on past the other process's
(`AcpConnectionPool.adopt`, `AcpClient.continue_ids_from`), the
prompt's answer awaited (`AcpClient.await_response`), the reply built
as a turn ended here is -- the partial notice when the agent stopped
early, the machine note saying it was picked up after this computer
restarted -- then the dispatch's record finished, the registry row and
the node claim closed, and the result announced into the conversation
that asked, as the manager announces a turn that ended here
(`raven/acp_client/remote_resume.py`). A machine that cannot be asked
keeps its record for a later start; a run the machine no longer has,
one whose agent ended while this computer was away, one whose agent
left the agent table, and one whose turn says nothing are announced as
errors that say so. The adopted agent process serves later dispatches
of its entry, as one started here would.
- Seams: `AcpClient.request(sent=)` hears the id of what goes out;
`ChannelEndpoint.watchers` hear the frames the connection has dealt
with (`consumed`, called by the read loop); `ChannelEndpoint.stays`
reads the leaving flag.
- A cli entry's run on a machine is recorded while it runs (kind `cli`,
no ACP part) and left the same way, the manager reading the dispatch
as left; the entry picks it up (`CliAgentBackend.pick_up`): the daemon
replays the command's whole output from its first line, the exit code
ends it, and the reply closes with the machine note.
- The record carries the task as the model wrote it, and -- once the
manager has read the run as left -- the dispatch's origin and summary
(`remote_runs.annotate`): the announce after a restart names the
instance, node and tool call as the announce here does and quotes the
task as authored, never the rendered prompt on disk; a host killed
outright gets an origin made up from the spawn record and the run.
- The record goes last, once the outcome has reached its conversation:
a conclusion that fails, a stop under it, or an announce the scheduler
refuses (`_announce_result` says so now) leaves the record, its claim
released, for the next start to try again.
- A run that returned here keeps its record the same way, until the
manager's announce of the reply was taken: the manager dispatches with
`reported_by_caller=True`, the backend then leaves the record behind
with the reply instead of removing it as it returns, and the manager
removes it after the announce (`remote_runs.forget_dispatch`) -- or,
when the draining scheduler refused the announce, annotates it with
the dispatch's origin and summary for the next start to report. A host
killed between the agent's answer and the announce therefore still has
the way back to the run. A caller that takes the reply as its own
result (a graph node, a probe, a direct chat) dispatches without the
keyword, and the record goes as it did. The record's frame count stops
at the prompt: the turn's own frames, the chunks before the answer as
much as the answer, are never checkpointed, so a pick-up replays the
turn whole and the reply it reports is complete; a record that had
moved past the answer would have resumed after it and waited forever.
Not in this change: a run from the DAG lane has no spawn record to
finish; it is announced from the origin the manager added to its record
when it was left, or, for a host killed outright, from the run's record
alone. The Tasks page reads the finished record on its next load rather
than live.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Machine availability and restart checks after removing the trial switch:
- New default-behavior regressions were run before the removal: 8
failed,
with the enabled-value control passing. After removal, all 9 passed,
including ACP/CLI recovery through the real AgentRegistry with the
variable absent or set to `0`.
- `uv run --no-sync pytest tests/test_acp_remote.py
tests/test_node_client.py tests/test_node_server.py
tests/test_node_install.py tests/test_ops_remote_path.py
tests/test_permissions_remote_read.py tests/test_remote_file_tools.py
tests/test_deliver_files_tool.py tests/test_raven_config_tool.py
tests/test_rpc_subagents.py tests/test_shell_machine_channel.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_resume_remote.py tests/test_remote_resume.py
tests/test_remote_runs.py tests/test_subagent_manager.py
tests/test_cli_tui_commands.py tests/test_rpc_schema_match.py
tests/test_config_schema.py -n 4 --idle-ceiling-strict --cov=raven
--cov-branch --cov-report=json:<coverage-json> --cov-report= --tb=short`
-- 1799 passed on the combined tree, including the latest published
result-retention fix. An earlier run had a missing stdio response in the
minimal-node dependency test and an idle-gate hit; both passed a focused
recheck, and this final complete selection passed with the strict gate.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 92.92% (656/706 changed executable lines) for this PR; the
gate-removal
commit alone covers 2/2 new executable lines (the remainder deletes
gates
and updates descriptions).
- `UV_NO_SYNC=1 make lint-python lint-imports lint-deps lint-types`, the
commit/source-language/large-file scope gates and `git diff --check`
pass.
Both RPC generators' `--check` runs pass; `npm test --
scripts/gates/fixture-shape.test.mjs`
from `ui-web` passes all five tests.
- Real Linux machine, isolated node installations, separate dispatch and
restart processes: before removal, a task started with the trial switch
on was reported as an absent agent and its recovery record deleted when
restarted without it. After removal, both absent and old `0` settings
recover the full CLI result and read its output file by address. The
absent-variable case kills the relay before host shutdown. The combined
final tree was reinstalled and both scenarios repeated successfully.
- With the variable absent, 12 further timing/boundary probes pass:
early
CLI exit and blank lines, relay loss before heartbeat with normal-call
recovery, tunnel loss, ACP/CLI/OpenAI cancellation during disconnection,
11 launches admitting 8 and refusing 3, occupied tunnel port, and an
unwritable prompt file. Three attachment, POSIX-command and IPv6 checks
also pass; Windows command parsing is simulated, not native Windows.
No test task or tunnel remains; the dedicated node directories are
removed.
- CONTEXT.md, configuration and RPC descriptions, transport/path
docstrings,
both collaboration guides and the owner's design notes were checked
against the code. The guides explain the former trial switch and normal
startup without it. The existing SSH, permission and limit tests
continue
to exercise their checks with no test-only switch setup.
- `env -u RAVEN_DEBUG_MACHINES uv run --no-sync pytest -n 4 --tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
-- 31345 passed, 230 skipped, 2 xfailed, 5 failed. A separate
current-main
checkout reproduces four failures: CLI color rendering on this host,
LibreOffice detection, and the two simulation cases. The remaining
failure
was the direct-chat test stub rejecting `reported_by_caller` from the
latest result-retention change; the published wrapper fix now forwards
only keywords declared by its fake backend. The full
suite was not rerun for that test-only correction. Final direct-chat and
manager recheck: 277 passed with strict idle accounting. Full-PR
changed-line coverage from this
run is 96.18% (679/706).
Earlier verification:
- `uv run pytest tests/test_remote_runs.py tests/test_remote_resume.py
tests/test_resume_remote.py tests/test_acp_client_resume.py
tests/test_node_client.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_cli_tui_commands.py
tests/test_subagent_manager.py tests/test_rpc_bootstrap.py
tests/test_cli_gateway_commands.py tests/test_acp_client_launch.py
tests/test_subagent_acp.py tests/test_scope_canon.py
tests/test_l4_entrances.py`
-- all passed (605 passed). Covered: the record written owner-only, kept
current
per frame and per prompt, removed when the turn ends here and kept when
this process leaves; a dispatch left at shutdown neither finished nor
announced by the manager; an endpoint ended while leaving leaves the
agent with the daemon (real daemon); a task picked up from the frame
last read, with the watchers hearing each frame; a pick-up the machine
refuses or cannot carry; an answer awaited for a request another
process sent, ids carried on, no `session/cancel` while leaving; the
reply read off replayed frames with the other sessions' frames
ignored, a turn cut short marked partial, an unreachable machine
keeping the record, a run the machine no longer has, a prompt never
sent, an agent that ended while this computer was away, an answer
waited out, a turn that said nothing, a pick-up beside a live
connection closing its own; the manager-side conclusion: record
finished and announced, an error announced and the record failed, an
unreachable machine keeping the record, an agent off the table, a
failure inside one pick-up not stopping the next, a run with no
conversation, the once-per-process scheduling; a cli run recorded
while it runs and left at shutdown, picked up whole on the real daemon
behind the stand-in ssh, its pick-up's errors said (a non-zero exit, a
machine that no longer has it, a link down, this computer leaving), a
cancel that is not a leave ending the command with no record left; the
TUI's stop saying the agents stay before the cancel and its start
scheduling the pick-up; the record annotated with the dispatch's
origin and summary when the manager reads it as left, the announce
quoting the task as authored and naming instance, node and tool call,
with and without the manager's fields; the record outliving a refused
announce and a failed conclusion, claim released; a cli record read
without an ACP session, an older record read with defaults; the record
moving with the frames before the prompt and not past it, a dispatch's
record forgotten by the manager's id with its claim, the manager
dispatching with `reported_by_caller` and forgetting the record after
an announce that was taken and annotating it after one that was
refused; on the real daemon behind the stand-in ssh, an acp reply and
a cli output returned with the record kept and the host "stopped"
before the announce, the next start reading the same reply again
whole, and a dispatch without the keyword leaving no record. End to end
on the real
daemon behind the stand-in ssh: a turn dispatched, this process
"stopped" the way the gateway stops, the agent still running on the
daemon, then a fresh pick-up reading its answer, finishing the record
and announcing it, with the agent's process the pool's connection
again.
- `uv run pytest` on this tree, at an earlier revision of this change --
29270 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine; diff coverage of this
change against its base: 97.45% (153 of 157 executable changed lines),
and 94.67% for the four changes against main; `lint-imports` 10 kept,
0 broken; `ty` clean.
- Real machine (a registered Linux machine with Claude Code signed in,
node reinstalled with this tree): process A dispatched a 25 s task,
saw the prompt out after 2.4 s and the record keep up with the frames
(6, then 10), then stopped the way the gateway stops 5.4 s after the
dispatch, leaving the record and the agent running there; process B,
started fresh, found the record and the running task, picked it up
in 2.4 s, announced the agent's reply to the conversation with the
machine note, finished the record, removed the run's record, held the
agent's process as the pool's connection, and closing the pool ended
the agent on the machine. This revision, the cli lane with the code
before and after, the node reinstalled for each: a cli entry
`sh -c 'echo $$ > pid.txt; sleep 20; echo done; cat'` was dispatched
and the host stopped the gateway's way 2.7 s later. Before: no
record, the command left running on the machine, and the next start
found nothing to pick up. After: the record kept with the task as
authored and the manager's origin, the command running on, and the
next start picked it up in 16.8 s (the sleep), announcing `done` and
`hi` with the instance, node and tool call and the machine note. The
same with the link killed before the stop: recorded, left, picked up.
With the machine's daemon ended between the stop and the pick-up, it
ended its groups on its way out, the task's record there said exit
-9, and the pick-up announced that as the run's error in 0.1 s,
leaving nothing on the machine.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint`,
`ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese agent-collaboration guides explain normal
startup,
registered-machine setup and recovery, and the removed trial switch.
## Risk
Configured machine entries are available by default; entries without a
`machine` remain local. A remote-looking file path is resolved against
the
registry even without an environment variable; use an explicit local
path
such as `./a:b` for a local filename containing a colon. A formerly
hidden
machine entry is now available for dispatch, and an old variable set to
`0`
no longer disables it. Stopping Raven no longer cancels a turn
running on a machine; it goes on there until the agent finishes or is
cancelled, and its answer arrives in the conversation after the next
start, as a sub-agent result that triggers a turn. One record per run in
the raven home, owner-only, removed when the run is concluded; a machine
that cannot be reached keeps it, so a run is reported at most once and
never lost quietly. The adopted connection answers requests with ids
past the ones the stopped process used, so a replayed frame resolves
nothing sent after the restart. Rollback is a plain revert; a record
left
behind is then an inert file.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#920 and #922 (merged); #921 was carried by #922.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-14 21:34:19 +08:00
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_node_starts_without_a_debug_switch(box, pool, monkeypatch):
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
box.node()
|
feat(*): pick up a turn left on a registered machine when this computer stops (#934)
## Summary
Registered machines are now available without a debug environment
switch.
`RAVEN_DEBUG_MACHINES` previously defaulted to off: it kept
machine-backed
agents out of the dispatch table, refused node/installer access, hid
remote
preset destinations, and treated file paths locally while off. It was a
trial feature gate, not a verbose-logging option. At the owner's
request,
this PR removes the variable and all of those checks. Normal startup and
restart recovery use the configured machine entries; even a leftover
value
of `0` is ignored. Machine registration, SSH authentication, file
permission
prompts, and concurrency limits still apply.
Fourth change in the series that lets a Raven dispatch work to the
machines in its machine registry. #920, #921 (merged through #922) and
#922 are on main; this change stands on main alone now, and the diff is
its own.
What this does: a turn running on a registered machine survives this
computer stopping, and the next start picks it up and reports it.
- While a turn runs on a machine, this computer remembers what the
machine does not know (`raven/acp_client/remote_runs.py`): one
owner-only file per run under `<raven home>/machines/runs/`, with the
machine's registry id, the task id its daemon runs the agent under, the
agent, the session key, the instance handle, the dispatch id, the ACP
session, the id of the prompt whose answer is awaited, the ids this
side has used, how many of the agent's frames were read here, the
agent's answer to `initialize` and the connection's launch key. Kept
current after every frame the connection has dealt with, removed when
the turn ends here. No address is in it.
- When this process stops -- the gateway, `serve`, the TUI, the page
stack -- the
agents on machines stay with their daemons
(`remote.leave_agents_running()` before the drain): a cancelled turn
on a machine sends no `session/cancel`, closing its connection ends the
endpoint only, and the manager leaves the dispatch's record open rather
than finishing it as cancelled. A hard kill or `/restart` leaves the
same record, written as the turn ran.
- At the next start -- the gateway once its generation is bound; the
`serve`, TUI and page stacks once --
`raven/agent/subagent/resume_remote.py`
picks every run up: `task/list` on the machine, `agent/attach` from the
frame last read (`NodePool.attach`), the connection adopted into the
pool with request ids carried on past the other process's
(`AcpConnectionPool.adopt`, `AcpClient.continue_ids_from`), the
prompt's answer awaited (`AcpClient.await_response`), the reply built
as a turn ended here is -- the partial notice when the agent stopped
early, the machine note saying it was picked up after this computer
restarted -- then the dispatch's record finished, the registry row and
the node claim closed, and the result announced into the conversation
that asked, as the manager announces a turn that ended here
(`raven/acp_client/remote_resume.py`). A machine that cannot be asked
keeps its record for a later start; a run the machine no longer has,
one whose agent ended while this computer was away, one whose agent
left the agent table, and one whose turn says nothing are announced as
errors that say so. The adopted agent process serves later dispatches
of its entry, as one started here would.
- Seams: `AcpClient.request(sent=)` hears the id of what goes out;
`ChannelEndpoint.watchers` hear the frames the connection has dealt
with (`consumed`, called by the read loop); `ChannelEndpoint.stays`
reads the leaving flag.
- A cli entry's run on a machine is recorded while it runs (kind `cli`,
no ACP part) and left the same way, the manager reading the dispatch
as left; the entry picks it up (`CliAgentBackend.pick_up`): the daemon
replays the command's whole output from its first line, the exit code
ends it, and the reply closes with the machine note.
- The record carries the task as the model wrote it, and -- once the
manager has read the run as left -- the dispatch's origin and summary
(`remote_runs.annotate`): the announce after a restart names the
instance, node and tool call as the announce here does and quotes the
task as authored, never the rendered prompt on disk; a host killed
outright gets an origin made up from the spawn record and the run.
- The record goes last, once the outcome has reached its conversation:
a conclusion that fails, a stop under it, or an announce the scheduler
refuses (`_announce_result` says so now) leaves the record, its claim
released, for the next start to try again.
- A run that returned here keeps its record the same way, until the
manager's announce of the reply was taken: the manager dispatches with
`reported_by_caller=True`, the backend then leaves the record behind
with the reply instead of removing it as it returns, and the manager
removes it after the announce (`remote_runs.forget_dispatch`) -- or,
when the draining scheduler refused the announce, annotates it with
the dispatch's origin and summary for the next start to report. A host
killed between the agent's answer and the announce therefore still has
the way back to the run. A caller that takes the reply as its own
result (a graph node, a probe, a direct chat) dispatches without the
keyword, and the record goes as it did. The record's frame count stops
at the prompt: the turn's own frames, the chunks before the answer as
much as the answer, are never checkpointed, so a pick-up replays the
turn whole and the reply it reports is complete; a record that had
moved past the answer would have resumed after it and waited forever.
Not in this change: a run from the DAG lane has no spawn record to
finish; it is announced from the origin the manager added to its record
when it was left, or, for a host killed outright, from the run's record
alone. The Tasks page reads the finished record on its next load rather
than live.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Machine availability and restart checks after removing the trial switch:
- New default-behavior regressions were run before the removal: 8
failed,
with the enabled-value control passing. After removal, all 9 passed,
including ACP/CLI recovery through the real AgentRegistry with the
variable absent or set to `0`.
- `uv run --no-sync pytest tests/test_acp_remote.py
tests/test_node_client.py tests/test_node_server.py
tests/test_node_install.py tests/test_ops_remote_path.py
tests/test_permissions_remote_read.py tests/test_remote_file_tools.py
tests/test_deliver_files_tool.py tests/test_raven_config_tool.py
tests/test_rpc_subagents.py tests/test_shell_machine_channel.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_resume_remote.py tests/test_remote_resume.py
tests/test_remote_runs.py tests/test_subagent_manager.py
tests/test_cli_tui_commands.py tests/test_rpc_schema_match.py
tests/test_config_schema.py -n 4 --idle-ceiling-strict --cov=raven
--cov-branch --cov-report=json:<coverage-json> --cov-report= --tb=short`
-- 1799 passed on the combined tree, including the latest published
result-retention fix. An earlier run had a missing stdio response in the
minimal-node dependency test and an idle-gate hit; both passed a focused
recheck, and this final complete selection passed with the strict gate.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 92.92% (656/706 changed executable lines) for this PR; the
gate-removal
commit alone covers 2/2 new executable lines (the remainder deletes
gates
and updates descriptions).
- `UV_NO_SYNC=1 make lint-python lint-imports lint-deps lint-types`, the
commit/source-language/large-file scope gates and `git diff --check`
pass.
Both RPC generators' `--check` runs pass; `npm test --
scripts/gates/fixture-shape.test.mjs`
from `ui-web` passes all five tests.
- Real Linux machine, isolated node installations, separate dispatch and
restart processes: before removal, a task started with the trial switch
on was reported as an absent agent and its recovery record deleted when
restarted without it. After removal, both absent and old `0` settings
recover the full CLI result and read its output file by address. The
absent-variable case kills the relay before host shutdown. The combined
final tree was reinstalled and both scenarios repeated successfully.
- With the variable absent, 12 further timing/boundary probes pass:
early
CLI exit and blank lines, relay loss before heartbeat with normal-call
recovery, tunnel loss, ACP/CLI/OpenAI cancellation during disconnection,
11 launches admitting 8 and refusing 3, occupied tunnel port, and an
unwritable prompt file. Three attachment, POSIX-command and IPv6 checks
also pass; Windows command parsing is simulated, not native Windows.
No test task or tunnel remains; the dedicated node directories are
removed.
- CONTEXT.md, configuration and RPC descriptions, transport/path
docstrings,
both collaboration guides and the owner's design notes were checked
against the code. The guides explain the former trial switch and normal
startup without it. The existing SSH, permission and limit tests
continue
to exercise their checks with no test-only switch setup.
- `env -u RAVEN_DEBUG_MACHINES uv run --no-sync pytest -n 4 --tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
-- 31345 passed, 230 skipped, 2 xfailed, 5 failed. A separate
current-main
checkout reproduces four failures: CLI color rendering on this host,
LibreOffice detection, and the two simulation cases. The remaining
failure
was the direct-chat test stub rejecting `reported_by_caller` from the
latest result-retention change; the published wrapper fix now forwards
only keywords declared by its fake backend. The full
suite was not rerun for that test-only correction. Final direct-chat and
manager recheck: 277 passed with strict idle accounting. Full-PR
changed-line coverage from this
run is 96.18% (679/706).
Earlier verification:
- `uv run pytest tests/test_remote_runs.py tests/test_remote_resume.py
tests/test_resume_remote.py tests/test_acp_client_resume.py
tests/test_node_client.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_cli_tui_commands.py
tests/test_subagent_manager.py tests/test_rpc_bootstrap.py
tests/test_cli_gateway_commands.py tests/test_acp_client_launch.py
tests/test_subagent_acp.py tests/test_scope_canon.py
tests/test_l4_entrances.py`
-- all passed (605 passed). Covered: the record written owner-only, kept
current
per frame and per prompt, removed when the turn ends here and kept when
this process leaves; a dispatch left at shutdown neither finished nor
announced by the manager; an endpoint ended while leaving leaves the
agent with the daemon (real daemon); a task picked up from the frame
last read, with the watchers hearing each frame; a pick-up the machine
refuses or cannot carry; an answer awaited for a request another
process sent, ids carried on, no `session/cancel` while leaving; the
reply read off replayed frames with the other sessions' frames
ignored, a turn cut short marked partial, an unreachable machine
keeping the record, a run the machine no longer has, a prompt never
sent, an agent that ended while this computer was away, an answer
waited out, a turn that said nothing, a pick-up beside a live
connection closing its own; the manager-side conclusion: record
finished and announced, an error announced and the record failed, an
unreachable machine keeping the record, an agent off the table, a
failure inside one pick-up not stopping the next, a run with no
conversation, the once-per-process scheduling; a cli run recorded
while it runs and left at shutdown, picked up whole on the real daemon
behind the stand-in ssh, its pick-up's errors said (a non-zero exit, a
machine that no longer has it, a link down, this computer leaving), a
cancel that is not a leave ending the command with no record left; the
TUI's stop saying the agents stay before the cancel and its start
scheduling the pick-up; the record annotated with the dispatch's
origin and summary when the manager reads it as left, the announce
quoting the task as authored and naming instance, node and tool call,
with and without the manager's fields; the record outliving a refused
announce and a failed conclusion, claim released; a cli record read
without an ACP session, an older record read with defaults; the record
moving with the frames before the prompt and not past it, a dispatch's
record forgotten by the manager's id with its claim, the manager
dispatching with `reported_by_caller` and forgetting the record after
an announce that was taken and annotating it after one that was
refused; on the real daemon behind the stand-in ssh, an acp reply and
a cli output returned with the record kept and the host "stopped"
before the announce, the next start reading the same reply again
whole, and a dispatch without the keyword leaving no record. End to end
on the real
daemon behind the stand-in ssh: a turn dispatched, this process
"stopped" the way the gateway stops, the agent still running on the
daemon, then a fresh pick-up reading its answer, finishing the record
and announcing it, with the agent's process the pool's connection
again.
- `uv run pytest` on this tree, at an earlier revision of this change --
29270 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine; diff coverage of this
change against its base: 97.45% (153 of 157 executable changed lines),
and 94.67% for the four changes against main; `lint-imports` 10 kept,
0 broken; `ty` clean.
- Real machine (a registered Linux machine with Claude Code signed in,
node reinstalled with this tree): process A dispatched a 25 s task,
saw the prompt out after 2.4 s and the record keep up with the frames
(6, then 10), then stopped the way the gateway stops 5.4 s after the
dispatch, leaving the record and the agent running there; process B,
started fresh, found the record and the running task, picked it up
in 2.4 s, announced the agent's reply to the conversation with the
machine note, finished the record, removed the run's record, held the
agent's process as the pool's connection, and closing the pool ended
the agent on the machine. This revision, the cli lane with the code
before and after, the node reinstalled for each: a cli entry
`sh -c 'echo $$ > pid.txt; sleep 20; echo done; cat'` was dispatched
and the host stopped the gateway's way 2.7 s later. Before: no
record, the command left running on the machine, and the next start
found nothing to pick up. After: the record kept with the task as
authored and the manager's origin, the command running on, and the
next start picked it up in 16.8 s (the sleep), announcing `done` and
`hi` with the instance, node and tool call and the machine note. The
same with the link killed before the stop: recorded, left, picked up.
With the machine's daemon ended between the stop and the pick-up, it
ended its groups on its way out, the task's record there said exit
-9, and the pick-up announced that as the run's error in 0.1 s,
leaving nothing on the machine.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint`,
`ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese agent-collaboration guides explain normal
startup,
registered-machine setup and recovery, and the removed trial switch.
## Risk
Configured machine entries are available by default; entries without a
`machine` remain local. A remote-looking file path is resolved against
the
registry even without an environment variable; use an explicit local
path
such as `./a:b` for a local filename containing a colon. A formerly
hidden
machine entry is now available for dispatch, and an old variable set to
`0`
no longer disables it. Stopping Raven no longer cancels a turn
running on a machine; it goes on there until the agent finishes or is
cancelled, and its answer arrives in the conversation after the next
start, as a sub-agent result that triggers a turn. One record per run in
the raven home, owner-only, removed when the run is concluded; a machine
that cannot be reached keeps it, so a run is reported at most once and
never lost quietly. The adopted connection answers requests with ids
past the ones the stopped process used, so a replayed frame resolves
nothing sent after the restart. Rollback is a plain revert; a record
left
behind is then an inert file.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#920 and #922 (merged); #921 was carried by #922.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-14 21:34:19 +08:00
|
|
|
monkeypatch.delenv("RAVEN_DEBUG_MACHINES", raising=False)
|
|
|
|
|
hello = await pool.hello("box")
|
|
|
|
|
assert hello["protocol"] == protocol.PROTOCOL
|
|
|
|
|
assert len(box.launches()) == 1
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_machine_that_is_not_registered_is_refused_before_any_ssh(box, pool):
|
|
|
|
|
with pytest.raises(NodeError, match="machine 'elsewhere' is not registered"):
|
|
|
|
|
await pool.hello("elsewhere")
|
|
|
|
|
|
|
|
|
|
assert box.launches() == []
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_the_ssh_argv_reaches_the_connection_as_it_is_for_no_host_grammar_to_read_again(box, pool, monkeypatch):
|
|
|
|
|
launched: list[Any] = []
|
|
|
|
|
|
|
|
|
|
class _Stub:
|
|
|
|
|
@classmethod
|
|
|
|
|
async def launch(cls, **kw: Any) -> None:
|
|
|
|
|
launched.append(kw["command"])
|
|
|
|
|
raise AcpConnectionError("stood in")
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(node_client, "AcpClient", _Stub)
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError, match="could not start ssh to machine 'Lab box'"):
|
|
|
|
|
await pool.hello("box")
|
|
|
|
|
|
|
|
|
|
(argv,) = launched
|
|
|
|
|
assert isinstance(argv, list) and argv[0] == "ssh" and argv[-2] == "--", "an argv, not a line to split again"
|
|
|
|
|
assert argv[-1].startswith("exec /bin/sh -c 'echo; exec ") and "-m raven.node --attach --dir" in argv[-1]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_an_ssh_command_that_cannot_be_made_is_one_sentence(box, pool, monkeypatch):
|
|
|
|
|
def refuse(*args: Any, **kwargs: Any) -> str:
|
|
|
|
|
raise ValueError("a command for a registered machine must be one line")
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(remote, "ssh_command", refuse)
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError, match=r"^machine 'Lab box' \(box\): a command .* must be one line$") as caught:
|
|
|
|
|
await pool.hello("box")
|
|
|
|
|
|
|
|
|
|
_no_address(str(caught.value))
|
|
|
|
|
assert box.launches() == []
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_no_ssh_on_this_computer_is_one_sentence(box, pool, monkeypatch, tmp_path):
|
|
|
|
|
empty = tmp_path / "empty-bin"
|
|
|
|
|
empty.mkdir()
|
|
|
|
|
monkeypatch.setattr(backend_env, "_LOGIN_ENV", {**os.environ, "PATH": str(empty)})
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError, match=r"^could not start ssh to machine 'Lab box' \(box\): ") as caught:
|
|
|
|
|
await pool.hello("box")
|
|
|
|
|
|
|
|
|
|
_no_address(str(caught.value))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- what the link carries beside file calls -----------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_the_session_directory_is_made_on_the_machine_and_comes_back_absolute(box, pool):
|
|
|
|
|
box.node()
|
|
|
|
|
|
|
|
|
|
made, since = await pool.session_dir("box", "~/raven work/a b-1234")
|
|
|
|
|
|
|
|
|
|
assert made == str((box.home / "raven work" / "a b-1234").resolve())
|
|
|
|
|
assert Path(made).is_dir()
|
|
|
|
|
assert abs(since - time.time()) < 60, "the machine's clock, as the directory was made"
|
|
|
|
|
assert len(box.launches()) == 1, "on the link, not an ssh of its own"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_session_directory_the_machine_cannot_make_is_refused_with_its_reason(box, pool):
|
|
|
|
|
box.node()
|
|
|
|
|
(box.home / "blocker").write_text("a file, not a directory\n", encoding="utf-8")
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError, match=r"refused session/dir: could not make .*blocker/t1") as caught:
|
|
|
|
|
await pool.session_dir("box", "~/blocker/t1")
|
|
|
|
|
|
|
|
|
|
_no_address(str(caught.value))
|
|
|
|
|
assert pool.live() == ["box"], "a refusal keeps the link"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_the_machine_and_an_agent_s_command_are_probed_over_the_link(box, pool):
|
|
|
|
|
box.node()
|
|
|
|
|
|
|
|
|
|
machine = await pool.probe_machine("box")
|
|
|
|
|
found = await pool.probe_agent("box", box.agent())
|
|
|
|
|
missing = await pool.probe_agent("box", "no-such-agent-for-raven-tests --acp")
|
|
|
|
|
|
|
|
|
|
assert machine["python"].count(".") == 2 and str(box.home) in machine["disk_free_kb"]
|
|
|
|
|
assert found["found"] == sys.executable and missing["found"] is None
|
|
|
|
|
assert len(box.launches()) == 1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- an agent on the machine, through the node -----------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_an_agent_started_through_the_node_is_driven_as_a_local_one_is(box, pool):
|
|
|
|
|
box.node()
|
|
|
|
|
work = box.home / "raven-work"
|
|
|
|
|
|
|
|
|
|
endpoint = await pool.open_channel("box", name="echo@box", command=box.agent(), cwd="~/raven-work", env={})
|
|
|
|
|
client = AcpClient.over(endpoint, name="echo@box")
|
|
|
|
|
try:
|
|
|
|
|
assert await client.request("ping", {"x": 1}, timeout=10) == {"echo": {"x": 1}}
|
|
|
|
|
assert await client.request("ping", {"x": 2}, timeout=10) == {"echo": {"x": 2}}
|
|
|
|
|
await _until(lambda: "agent: started" in client.stderr_tail())
|
|
|
|
|
assert work.is_dir(), "the agent's root is made for it"
|
|
|
|
|
assert isinstance(endpoint.pid, int) and endpoint.describe().startswith("task echo@box-")
|
|
|
|
|
assert "on machine 'Lab box' (box), pid " in endpoint.describe()
|
|
|
|
|
assert pool.status("box").state == "connected" and pool.status("box").said() == "connected, 1 agent running"
|
|
|
|
|
assert len(box.launches()) == 1, "the agent rides the node's ssh"
|
|
|
|
|
finally:
|
|
|
|
|
await client.close()
|
|
|
|
|
await _until(lambda: pool.status("box").state == "idle")
|
|
|
|
|
assert pool.status("box").said() == "connected, idle"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_the_entry_s_env_reaches_the_agent_over_the_link_and_never_an_ssh_argument(box, pool):
|
|
|
|
|
box.node()
|
|
|
|
|
probe = (
|
|
|
|
|
"import json, os, sys; "
|
|
|
|
|
"print(json.dumps({'jsonrpc': '2.0', 'id': 1, 'result': {'key': os.environ.get('OPENAI_API_KEY'), "
|
|
|
|
|
"'role': os.environ.get('RAVEN_SUBAGENT')}}), flush=True); sys.stdin.read()"
|
|
|
|
|
)
|
|
|
|
|
env = {"OPENAI_API_KEY": "sk-raven-test-not-real", "RAVEN_SUBAGENT": "1"}
|
|
|
|
|
|
|
|
|
|
endpoint = await pool.open_channel(
|
|
|
|
|
"box", name="probe@box", command=f"{sys.executable} -c {json.dumps(probe)}", cwd="~", env=env
|
|
|
|
|
)
|
|
|
|
|
client = AcpClient.over(endpoint, name="probe@box")
|
|
|
|
|
try:
|
|
|
|
|
assert await client.request("whatever", {}, timeout=10) == {"key": "sk-raven-test-not-real", "role": "1"}
|
|
|
|
|
finally:
|
|
|
|
|
await client.close()
|
|
|
|
|
for argv in box.launches():
|
|
|
|
|
assert "sk-raven-test-not-real" not in " ".join(argv)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_an_agent_that_exits_ends_its_connection_with_its_exit_code(box, pool):
|
|
|
|
|
box.node()
|
|
|
|
|
endpoint = await pool.open_channel("box", name="echo@box", command=box.agent(), cwd="~", env={})
|
|
|
|
|
client = AcpClient.over(endpoint, name="echo@box")
|
|
|
|
|
|
|
|
|
|
with pytest.raises(AcpConnectionError, match=r"connection ended \(exit 7\)"):
|
|
|
|
|
await client.request("quit", {"rc": 7}, timeout=10)
|
|
|
|
|
|
|
|
|
|
await _until(lambda: not client.alive)
|
|
|
|
|
assert endpoint.returncode == 7 and endpoint.ended() is None
|
|
|
|
|
assert "agent: started" in client.stderr_tail()
|
|
|
|
|
assert pool.status("box").state == "idle", "the channel is gone, the link stays"
|
|
|
|
|
await client.close()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_closing_the_connection_ends_the_agent_and_what_it_started(box, pool):
|
|
|
|
|
box.node()
|
|
|
|
|
endpoint = await pool.open_channel("box", name="echo@box", command=box.agent(), cwd="~", env={})
|
|
|
|
|
client = AcpClient.over(endpoint, name="echo@box")
|
|
|
|
|
child = (await client.request("spawn", {}, timeout=10))["pid"]
|
|
|
|
|
agent = endpoint.pid
|
|
|
|
|
assert agent is not None and not _gone(agent) and not _gone(child)
|
|
|
|
|
|
|
|
|
|
await client.close()
|
|
|
|
|
|
|
|
|
|
await _until(lambda: _gone(agent))
|
|
|
|
|
await _until(lambda: _gone(child))
|
|
|
|
|
assert endpoint.returncode is not None and not client.alive
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_an_agent_whose_command_is_not_on_the_machine_says_so(box, pool):
|
|
|
|
|
box.node()
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError) as caught:
|
|
|
|
|
await pool.open_channel("box", name="x@box", command="no-such-agent-for-raven-tests --acp", cwd="~", env={})
|
|
|
|
|
|
|
|
|
|
assert str(caught.value) == (
|
|
|
|
|
"the agent's command was not found on machine 'Lab box' (box); "
|
|
|
|
|
"install it there, for the user the machine is registered with"
|
|
|
|
|
)
|
|
|
|
|
assert pool.live() == ["box"] and pool.status("box").state == "idle"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_an_agent_whose_root_cannot_be_made_says_so(box, pool):
|
|
|
|
|
box.node()
|
|
|
|
|
(box.home / "blocker").write_text("a file\n", encoding="utf-8")
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError, match=r"could not start 'x@box': could not make .*blocker/agents") as caught:
|
|
|
|
|
await pool.open_channel("box", name="x@box", command=box.agent(), cwd="~/blocker/agents", env={})
|
|
|
|
|
|
|
|
|
|
_no_address(str(caught.value))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_machine_full_of_agents_refuses_the_next_in_one_sentence(tmp_path):
|
|
|
|
|
# The node's own refusal (tested with the node); here the sentence the
|
|
|
|
|
# host makes of its code, with the limit the handshake reported.
|
|
|
|
|
pool = NodePool()
|
|
|
|
|
busy = AcpRemoteError(protocol.AGENT_LAUNCH, protocol.BUSY, "8 agents are already running on this machine")
|
|
|
|
|
_live(pool, tmp_path, raises=busy)
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError, match=r"^machine 'Lab box' \(box\) already runs 8 agents; wait for one to finish$"):
|
|
|
|
|
await pool.open_channel("box", name="b@box", command="agent", cwd="~", env={})
|
|
|
|
|
|
|
|
|
|
assert pool.live() == ["box"] and pool._nodes["box"].channels == {}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_link_that_goes_leaves_the_agent_running_and_the_turn_goes_on_once_it_is_back(box, pool):
|
|
|
|
|
"""The ssh is the way to the machine, not the agent's life: a request made while it is down is answered after."""
|
|
|
|
|
box.node()
|
|
|
|
|
pool._heartbeat_s = 0.1
|
|
|
|
|
pool._lost_after_s = 1.0
|
|
|
|
|
pool._backoff_s = (0.2,)
|
|
|
|
|
endpoint = await pool.open_channel("box", name="echo@box", command=box.agent(), cwd="~", env={})
|
|
|
|
|
client = AcpClient.over(endpoint, name="echo@box")
|
|
|
|
|
assert await client.request("ping", {"x": 1}, timeout=10) == {"echo": {"x": 1}}
|
|
|
|
|
relay = pool._nodes["box"].client._proc.pid
|
|
|
|
|
daemon = pool._nodes["box"].hello["pid"]
|
|
|
|
|
assert relay != daemon, "the ssh's far end is the relay; the daemon is another process"
|
|
|
|
|
agent = endpoint.pid
|
|
|
|
|
|
|
|
|
|
os.kill(relay, signal.SIGKILL)
|
|
|
|
|
await _until(lambda: endpoint.detached is not None)
|
|
|
|
|
|
|
|
|
|
# Asked while the link is down: held here, sent once the task is attached again.
|
|
|
|
|
assert await client.request("ping", {"x": 2}, timeout=20) == {"echo": {"x": 2}}
|
|
|
|
|
assert endpoint.detached is None and endpoint.pid == agent and not _gone(agent)
|
|
|
|
|
assert len(box.launches()) == 2, "one ssh per link"
|
|
|
|
|
assert pool.status("box").state == "connected" and pool._nodes["box"].hello["pid"] == daemon
|
|
|
|
|
await client.close()
|
|
|
|
|
await _until(lambda: _gone(agent))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_daemon_that_stops_takes_its_agents_and_the_turn_says_so(box, pool):
|
|
|
|
|
box.node()
|
|
|
|
|
pool._heartbeat_s = 0.1
|
|
|
|
|
pool._lost_after_s = 1.0
|
|
|
|
|
pool._backoff_s = (0.2,)
|
|
|
|
|
endpoint = await pool.open_channel("box", name="echo@box", command=box.agent(), cwd="~", env={})
|
|
|
|
|
client = AcpClient.over(endpoint, name="echo@box")
|
|
|
|
|
assert await client.request("ping", {"x": 1}, timeout=10) == {"echo": {"x": 1}}
|
|
|
|
|
daemon = pool._nodes["box"].hello["pid"]
|
|
|
|
|
agent = endpoint.pid
|
|
|
|
|
|
|
|
|
|
os.kill(daemon, signal.SIGTERM)
|
|
|
|
|
|
|
|
|
|
await _until(lambda: _gone(agent)), "a daemon told to stop takes its agents with it"
|
|
|
|
|
with pytest.raises(AcpConnectionError) as caught:
|
|
|
|
|
await client.request("ping", {"x": 2}, timeout=20)
|
|
|
|
|
said = str(caught.value)
|
|
|
|
|
assert "echo@box" in said and "connection ended" in said
|
|
|
|
|
_no_address(said)
|
|
|
|
|
assert endpoint.returncode is not None and endpoint.returncode != 0
|
|
|
|
|
# The machine is reachable again: the next relay started a daemon anew.
|
|
|
|
|
assert pool.status("box").state in ("idle", "connected") and pool._nodes["box"].hello["pid"] != daemon
|
|
|
|
|
await client.close()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_closing_the_link_on_purpose_leaves_the_agent_on_the_machine_for_the_next_link_to_find(box, pool):
|
|
|
|
|
box.node()
|
|
|
|
|
endpoint = await pool.open_channel("box", name="echo@box", command=box.agent(), cwd="~", env={})
|
|
|
|
|
client = AcpClient.over(endpoint, name="echo@box")
|
|
|
|
|
assert await client.request("ping", {}, timeout=10) == {"echo": {}}
|
|
|
|
|
agent = endpoint.pid
|
|
|
|
|
|
|
|
|
|
await pool.close("box")
|
|
|
|
|
|
|
|
|
|
assert endpoint.ended() == "the link to machine 'Lab box' (box) was closed"
|
|
|
|
|
assert not _gone(agent), "closing the link is not cancelling the agent"
|
|
|
|
|
listed = await pool.tasks("box")
|
|
|
|
|
assert [(row["task"], row["state"], row["attached"]) for row in listed] == [(endpoint.task, "running", False)]
|
|
|
|
|
await pool.call("box", protocol.AGENT_ATTACH, {"channel": 9, "task": endpoint.task, "after": 1})
|
|
|
|
|
await pool.call("box", protocol.AGENT_CANCEL, {"channel": 9})
|
|
|
|
|
await _until(lambda: _gone(agent))
|
|
|
|
|
await client.close()
|
|
|
|
|
|
|
|
|
|
|
feat(*): pick up a turn left on a registered machine when this computer stops (#934)
## Summary
Registered machines are now available without a debug environment
switch.
`RAVEN_DEBUG_MACHINES` previously defaulted to off: it kept
machine-backed
agents out of the dispatch table, refused node/installer access, hid
remote
preset destinations, and treated file paths locally while off. It was a
trial feature gate, not a verbose-logging option. At the owner's
request,
this PR removes the variable and all of those checks. Normal startup and
restart recovery use the configured machine entries; even a leftover
value
of `0` is ignored. Machine registration, SSH authentication, file
permission
prompts, and concurrency limits still apply.
Fourth change in the series that lets a Raven dispatch work to the
machines in its machine registry. #920, #921 (merged through #922) and
#922 are on main; this change stands on main alone now, and the diff is
its own.
What this does: a turn running on a registered machine survives this
computer stopping, and the next start picks it up and reports it.
- While a turn runs on a machine, this computer remembers what the
machine does not know (`raven/acp_client/remote_runs.py`): one
owner-only file per run under `<raven home>/machines/runs/`, with the
machine's registry id, the task id its daemon runs the agent under, the
agent, the session key, the instance handle, the dispatch id, the ACP
session, the id of the prompt whose answer is awaited, the ids this
side has used, how many of the agent's frames were read here, the
agent's answer to `initialize` and the connection's launch key. Kept
current after every frame the connection has dealt with, removed when
the turn ends here. No address is in it.
- When this process stops -- the gateway, `serve`, the TUI, the page
stack -- the
agents on machines stay with their daemons
(`remote.leave_agents_running()` before the drain): a cancelled turn
on a machine sends no `session/cancel`, closing its connection ends the
endpoint only, and the manager leaves the dispatch's record open rather
than finishing it as cancelled. A hard kill or `/restart` leaves the
same record, written as the turn ran.
- At the next start -- the gateway once its generation is bound; the
`serve`, TUI and page stacks once --
`raven/agent/subagent/resume_remote.py`
picks every run up: `task/list` on the machine, `agent/attach` from the
frame last read (`NodePool.attach`), the connection adopted into the
pool with request ids carried on past the other process's
(`AcpConnectionPool.adopt`, `AcpClient.continue_ids_from`), the
prompt's answer awaited (`AcpClient.await_response`), the reply built
as a turn ended here is -- the partial notice when the agent stopped
early, the machine note saying it was picked up after this computer
restarted -- then the dispatch's record finished, the registry row and
the node claim closed, and the result announced into the conversation
that asked, as the manager announces a turn that ended here
(`raven/acp_client/remote_resume.py`). A machine that cannot be asked
keeps its record for a later start; a run the machine no longer has,
one whose agent ended while this computer was away, one whose agent
left the agent table, and one whose turn says nothing are announced as
errors that say so. The adopted agent process serves later dispatches
of its entry, as one started here would.
- Seams: `AcpClient.request(sent=)` hears the id of what goes out;
`ChannelEndpoint.watchers` hear the frames the connection has dealt
with (`consumed`, called by the read loop); `ChannelEndpoint.stays`
reads the leaving flag.
- A cli entry's run on a machine is recorded while it runs (kind `cli`,
no ACP part) and left the same way, the manager reading the dispatch
as left; the entry picks it up (`CliAgentBackend.pick_up`): the daemon
replays the command's whole output from its first line, the exit code
ends it, and the reply closes with the machine note.
- The record carries the task as the model wrote it, and -- once the
manager has read the run as left -- the dispatch's origin and summary
(`remote_runs.annotate`): the announce after a restart names the
instance, node and tool call as the announce here does and quotes the
task as authored, never the rendered prompt on disk; a host killed
outright gets an origin made up from the spawn record and the run.
- The record goes last, once the outcome has reached its conversation:
a conclusion that fails, a stop under it, or an announce the scheduler
refuses (`_announce_result` says so now) leaves the record, its claim
released, for the next start to try again.
- A run that returned here keeps its record the same way, until the
manager's announce of the reply was taken: the manager dispatches with
`reported_by_caller=True`, the backend then leaves the record behind
with the reply instead of removing it as it returns, and the manager
removes it after the announce (`remote_runs.forget_dispatch`) -- or,
when the draining scheduler refused the announce, annotates it with
the dispatch's origin and summary for the next start to report. A host
killed between the agent's answer and the announce therefore still has
the way back to the run. A caller that takes the reply as its own
result (a graph node, a probe, a direct chat) dispatches without the
keyword, and the record goes as it did. The record's frame count stops
at the prompt: the turn's own frames, the chunks before the answer as
much as the answer, are never checkpointed, so a pick-up replays the
turn whole and the reply it reports is complete; a record that had
moved past the answer would have resumed after it and waited forever.
Not in this change: a run from the DAG lane has no spawn record to
finish; it is announced from the origin the manager added to its record
when it was left, or, for a host killed outright, from the run's record
alone. The Tasks page reads the finished record on its next load rather
than live.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Machine availability and restart checks after removing the trial switch:
- New default-behavior regressions were run before the removal: 8
failed,
with the enabled-value control passing. After removal, all 9 passed,
including ACP/CLI recovery through the real AgentRegistry with the
variable absent or set to `0`.
- `uv run --no-sync pytest tests/test_acp_remote.py
tests/test_node_client.py tests/test_node_server.py
tests/test_node_install.py tests/test_ops_remote_path.py
tests/test_permissions_remote_read.py tests/test_remote_file_tools.py
tests/test_deliver_files_tool.py tests/test_raven_config_tool.py
tests/test_rpc_subagents.py tests/test_shell_machine_channel.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_resume_remote.py tests/test_remote_resume.py
tests/test_remote_runs.py tests/test_subagent_manager.py
tests/test_cli_tui_commands.py tests/test_rpc_schema_match.py
tests/test_config_schema.py -n 4 --idle-ceiling-strict --cov=raven
--cov-branch --cov-report=json:<coverage-json> --cov-report= --tb=short`
-- 1799 passed on the combined tree, including the latest published
result-retention fix. An earlier run had a missing stdio response in the
minimal-node dependency test and an idle-gate hit; both passed a focused
recheck, and this final complete selection passed with the strict gate.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 92.92% (656/706 changed executable lines) for this PR; the
gate-removal
commit alone covers 2/2 new executable lines (the remainder deletes
gates
and updates descriptions).
- `UV_NO_SYNC=1 make lint-python lint-imports lint-deps lint-types`, the
commit/source-language/large-file scope gates and `git diff --check`
pass.
Both RPC generators' `--check` runs pass; `npm test --
scripts/gates/fixture-shape.test.mjs`
from `ui-web` passes all five tests.
- Real Linux machine, isolated node installations, separate dispatch and
restart processes: before removal, a task started with the trial switch
on was reported as an absent agent and its recovery record deleted when
restarted without it. After removal, both absent and old `0` settings
recover the full CLI result and read its output file by address. The
absent-variable case kills the relay before host shutdown. The combined
final tree was reinstalled and both scenarios repeated successfully.
- With the variable absent, 12 further timing/boundary probes pass:
early
CLI exit and blank lines, relay loss before heartbeat with normal-call
recovery, tunnel loss, ACP/CLI/OpenAI cancellation during disconnection,
11 launches admitting 8 and refusing 3, occupied tunnel port, and an
unwritable prompt file. Three attachment, POSIX-command and IPv6 checks
also pass; Windows command parsing is simulated, not native Windows.
No test task or tunnel remains; the dedicated node directories are
removed.
- CONTEXT.md, configuration and RPC descriptions, transport/path
docstrings,
both collaboration guides and the owner's design notes were checked
against the code. The guides explain the former trial switch and normal
startup without it. The existing SSH, permission and limit tests
continue
to exercise their checks with no test-only switch setup.
- `env -u RAVEN_DEBUG_MACHINES uv run --no-sync pytest -n 4 --tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
-- 31345 passed, 230 skipped, 2 xfailed, 5 failed. A separate
current-main
checkout reproduces four failures: CLI color rendering on this host,
LibreOffice detection, and the two simulation cases. The remaining
failure
was the direct-chat test stub rejecting `reported_by_caller` from the
latest result-retention change; the published wrapper fix now forwards
only keywords declared by its fake backend. The full
suite was not rerun for that test-only correction. Final direct-chat and
manager recheck: 277 passed with strict idle accounting. Full-PR
changed-line coverage from this
run is 96.18% (679/706).
Earlier verification:
- `uv run pytest tests/test_remote_runs.py tests/test_remote_resume.py
tests/test_resume_remote.py tests/test_acp_client_resume.py
tests/test_node_client.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_cli_tui_commands.py
tests/test_subagent_manager.py tests/test_rpc_bootstrap.py
tests/test_cli_gateway_commands.py tests/test_acp_client_launch.py
tests/test_subagent_acp.py tests/test_scope_canon.py
tests/test_l4_entrances.py`
-- all passed (605 passed). Covered: the record written owner-only, kept
current
per frame and per prompt, removed when the turn ends here and kept when
this process leaves; a dispatch left at shutdown neither finished nor
announced by the manager; an endpoint ended while leaving leaves the
agent with the daemon (real daemon); a task picked up from the frame
last read, with the watchers hearing each frame; a pick-up the machine
refuses or cannot carry; an answer awaited for a request another
process sent, ids carried on, no `session/cancel` while leaving; the
reply read off replayed frames with the other sessions' frames
ignored, a turn cut short marked partial, an unreachable machine
keeping the record, a run the machine no longer has, a prompt never
sent, an agent that ended while this computer was away, an answer
waited out, a turn that said nothing, a pick-up beside a live
connection closing its own; the manager-side conclusion: record
finished and announced, an error announced and the record failed, an
unreachable machine keeping the record, an agent off the table, a
failure inside one pick-up not stopping the next, a run with no
conversation, the once-per-process scheduling; a cli run recorded
while it runs and left at shutdown, picked up whole on the real daemon
behind the stand-in ssh, its pick-up's errors said (a non-zero exit, a
machine that no longer has it, a link down, this computer leaving), a
cancel that is not a leave ending the command with no record left; the
TUI's stop saying the agents stay before the cancel and its start
scheduling the pick-up; the record annotated with the dispatch's
origin and summary when the manager reads it as left, the announce
quoting the task as authored and naming instance, node and tool call,
with and without the manager's fields; the record outliving a refused
announce and a failed conclusion, claim released; a cli record read
without an ACP session, an older record read with defaults; the record
moving with the frames before the prompt and not past it, a dispatch's
record forgotten by the manager's id with its claim, the manager
dispatching with `reported_by_caller` and forgetting the record after
an announce that was taken and annotating it after one that was
refused; on the real daemon behind the stand-in ssh, an acp reply and
a cli output returned with the record kept and the host "stopped"
before the announce, the next start reading the same reply again
whole, and a dispatch without the keyword leaving no record. End to end
on the real
daemon behind the stand-in ssh: a turn dispatched, this process
"stopped" the way the gateway stops, the agent still running on the
daemon, then a fresh pick-up reading its answer, finishing the record
and announcing it, with the agent's process the pool's connection
again.
- `uv run pytest` on this tree, at an earlier revision of this change --
29270 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine; diff coverage of this
change against its base: 97.45% (153 of 157 executable changed lines),
and 94.67% for the four changes against main; `lint-imports` 10 kept,
0 broken; `ty` clean.
- Real machine (a registered Linux machine with Claude Code signed in,
node reinstalled with this tree): process A dispatched a 25 s task,
saw the prompt out after 2.4 s and the record keep up with the frames
(6, then 10), then stopped the way the gateway stops 5.4 s after the
dispatch, leaving the record and the agent running there; process B,
started fresh, found the record and the running task, picked it up
in 2.4 s, announced the agent's reply to the conversation with the
machine note, finished the record, removed the run's record, held the
agent's process as the pool's connection, and closing the pool ended
the agent on the machine. This revision, the cli lane with the code
before and after, the node reinstalled for each: a cli entry
`sh -c 'echo $$ > pid.txt; sleep 20; echo done; cat'` was dispatched
and the host stopped the gateway's way 2.7 s later. Before: no
record, the command left running on the machine, and the next start
found nothing to pick up. After: the record kept with the task as
authored and the manager's origin, the command running on, and the
next start picked it up in 16.8 s (the sleep), announcing `done` and
`hi` with the instance, node and tool call and the machine note. The
same with the link killed before the stop: recorded, left, picked up.
With the machine's daemon ended between the stop and the pick-up, it
ended its groups on its way out, the task's record there said exit
-9, and the pick-up announced that as the run's error in 0.1 s,
leaving nothing on the machine.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint`,
`ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese agent-collaboration guides explain normal
startup,
registered-machine setup and recovery, and the removed trial switch.
## Risk
Configured machine entries are available by default; entries without a
`machine` remain local. A remote-looking file path is resolved against
the
registry even without an environment variable; use an explicit local
path
such as `./a:b` for a local filename containing a colon. A formerly
hidden
machine entry is now available for dispatch, and an old variable set to
`0`
no longer disables it. Stopping Raven no longer cancels a turn
running on a machine; it goes on there until the agent finishes or is
cancelled, and its answer arrives in the conversation after the next
start, as a sub-agent result that triggers a turn. One record per run in
the raven home, owner-only, removed when the run is concluded; a machine
that cannot be reached keeps it, so a run is reported at most once and
never lost quietly. The adopted connection answers requests with ids
past the ones the stopped process used, so a replayed frame resolves
nothing sent after the restart. Rollback is a plain revert; a record
left
behind is then an inert file.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#920 and #922 (merged); #921 was carried by #922.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-14 21:34:19 +08:00
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_while_this_computer_is_leaving_the_endpoint_s_end_leaves_the_agent_with_the_machine(box, pool):
|
|
|
|
|
"""The host is stopping: closing the connection ends the endpoint only, and the daemon keeps the agent."""
|
|
|
|
|
box.node()
|
|
|
|
|
endpoint = await pool.open_channel("box", name="echo@box", command=box.agent(), cwd="~", env={})
|
|
|
|
|
client = AcpClient.over(endpoint, name="echo@box")
|
|
|
|
|
assert await client.request("ping", {}, timeout=10) == {"echo": {}}
|
|
|
|
|
agent = endpoint.pid
|
|
|
|
|
|
|
|
|
|
remote.leave_agents_running()
|
|
|
|
|
try:
|
|
|
|
|
await client.close()
|
|
|
|
|
finally:
|
|
|
|
|
remote.staying()
|
|
|
|
|
|
|
|
|
|
assert endpoint.returncode == node_client.LOST_RC and "left running on task" in (endpoint.ended() or "")
|
|
|
|
|
await asyncio.sleep(0.3)
|
|
|
|
|
assert not _gone(agent), "the agent stays with the daemon"
|
|
|
|
|
assert [(row["task"], row["state"]) for row in await pool.tasks("box")] == [(endpoint.task, "running")]
|
|
|
|
|
picked, answer = await pool.attach("box", endpoint.task, name="echo@box", after=endpoint.received)
|
|
|
|
|
assert answer["pid"] == agent and answer["rc"] is None
|
|
|
|
|
await picked.end()
|
|
|
|
|
await _until(lambda: _gone(agent))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_task_is_picked_up_from_the_frame_last_read_and_the_watchers_hear_each_frame_dealt_with(box, pool):
|
|
|
|
|
box.node()
|
|
|
|
|
endpoint = await pool.open_channel("box", name="echo@box", command=box.agent(), cwd="~", env={})
|
|
|
|
|
heard: list[int] = []
|
|
|
|
|
endpoint.watchers.append(heard.append)
|
|
|
|
|
client = AcpClient.over(endpoint, name="echo@box")
|
|
|
|
|
await client.request("ping", {"n": 1}, timeout=10)
|
|
|
|
|
await client.request("ping", {"n": 2}, timeout=10)
|
|
|
|
|
await asyncio.sleep(0.05)
|
|
|
|
|
assert heard == [1, 2] and endpoint.received == 2
|
|
|
|
|
agent = endpoint.pid
|
|
|
|
|
await pool.close("box")
|
|
|
|
|
|
|
|
|
|
picked, answer = await pool.attach("box", endpoint.task, name="echo@box", after=2)
|
|
|
|
|
|
|
|
|
|
assert answer["pid"] == agent and answer["replayed"] == 0 and answer["rc"] is None
|
|
|
|
|
again = AcpClient.over(picked, name="echo@box")
|
|
|
|
|
again.continue_ids_from(100)
|
|
|
|
|
assert await again.request("ping", {"n": 3}, timeout=10) == {"echo": {"n": 3}}
|
|
|
|
|
assert picked.received == 3, "numbered on from the frames the first link read"
|
|
|
|
|
await again.close()
|
|
|
|
|
await _until(lambda: _gone(agent))
|
|
|
|
|
await client.close()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_picking_up_a_task_the_machine_does_not_know_is_one_sentence(box, pool):
|
|
|
|
|
box.node()
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError, match=r"machine 'Lab box' \(box\) no longer runs 'echo@box' \(task nothing-here\)"):
|
|
|
|
|
await pool.attach("box", "nothing-here", name="echo@box")
|
|
|
|
|
|
|
|
|
|
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
# --- a live node's calls, with the connection stood in for -------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class _Client:
|
|
|
|
|
"""A connection that answers, or fails, the way it is told to."""
|
|
|
|
|
|
|
|
|
|
def __init__(self, *, result: Any = None, raises: BaseException | None = None) -> None:
|
|
|
|
|
self.result = {"text": "ok"} if result is None else result
|
|
|
|
|
self.raises = raises
|
|
|
|
|
self.alive = True
|
|
|
|
|
self.asked: list[tuple[str, dict[str, Any], float]] = []
|
|
|
|
|
self.said: list[tuple[str, dict[str, Any]]] = []
|
|
|
|
|
|
|
|
|
|
async def request(self, method: str, params: dict[str, Any], *, timeout: float) -> Any:
|
|
|
|
|
self.asked.append((method, params, timeout))
|
|
|
|
|
if self.raises is not None and method != protocol.SHUTDOWN:
|
|
|
|
|
raise self.raises
|
|
|
|
|
return self.result
|
|
|
|
|
|
|
|
|
|
async def notify(self, method: str, params: dict[str, Any]) -> None:
|
|
|
|
|
self.said.append((method, params))
|
|
|
|
|
|
|
|
|
|
async def close(self) -> None:
|
|
|
|
|
self.alive = False
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _box(machine_id: str = "box") -> remote.Machine:
|
|
|
|
|
return remote.Machine(id=machine_id, display_name="Lab box", _row=row(id=machine_id))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _live(pool: NodePool, tmp_path: Path, machine_id: str = "box", client: _Client | None = None, **kw: Any) -> _Client:
|
|
|
|
|
client = client or _Client(**kw)
|
|
|
|
|
log = tmp_path / f"{machine_id}.log"
|
|
|
|
|
log.write_text(f"ssh: connect to host {HOST} port {PORT}: Connection refused\n", encoding="utf-8")
|
|
|
|
|
hello = {"protocol": protocol.PROTOCOL, "home": "/home/worker", "version": "0.9", "limits": {"agents": 8}}
|
|
|
|
|
pool._nodes[machine_id] = _Node(client, _box(machine_id), log, hello, {}) # type: ignore[arg-type]
|
|
|
|
|
return client
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_call_is_sent_with_its_budget_and_its_answer_returned(tmp_path):
|
|
|
|
|
pool = NodePool()
|
|
|
|
|
client = _live(pool, tmp_path, result={"text": "line 1"})
|
|
|
|
|
before = pool._nodes["box"].used
|
|
|
|
|
|
|
|
|
|
assert await pool.call("box", protocol.READ, {"path": "a"}, timeout=7.0) == {"text": "line 1"}
|
|
|
|
|
|
|
|
|
|
assert client.asked == [(protocol.READ, {"path": "a"}, 7.0)]
|
|
|
|
|
assert pool._nodes["box"].used >= before
|
|
|
|
|
assert await pool.home("box") == "/home/worker"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_an_answer_that_is_not_an_object_reads_as_empty(tmp_path):
|
|
|
|
|
pool = NodePool()
|
|
|
|
|
_live(pool, tmp_path, result=["not", "an", "object"])
|
|
|
|
|
|
|
|
|
|
assert await pool.call("box", protocol.READ, {}) == {}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_refused_call_keeps_the_node(tmp_path):
|
|
|
|
|
pool = NodePool()
|
|
|
|
|
client = _live(pool, tmp_path, raises=AcpRemoteError(protocol.READ, -32602, "roots must be a non-empty list"))
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError) as caught:
|
|
|
|
|
await pool.call("box", protocol.READ, {})
|
|
|
|
|
|
|
|
|
|
assert str(caught.value) == (
|
|
|
|
|
"the Raven node on machine 'Lab box' (box) refused fs/read: roots must be a non-empty list"
|
|
|
|
|
)
|
|
|
|
|
assert caught.value.reason is None and client.alive and pool.live() == ["box"]
|
|
|
|
|
client.raises = AcpRemoteError(protocol.WRITE, protocol.CHANGED, "changed")
|
|
|
|
|
with pytest.raises(NodeError) as changed:
|
|
|
|
|
await pool.call("box", protocol.WRITE, {})
|
|
|
|
|
assert changed.value.reason == "changed", "the node's own refusal, by name, for the file tools"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_call_that_timed_out_drops_the_node(tmp_path):
|
|
|
|
|
pool = NodePool(backoff_s=())
|
|
|
|
|
client = _live(pool, tmp_path, raises=AcpTimeoutError("no answer", method=protocol.READ))
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError, match=r"did not answer fs/read within 120s$"):
|
|
|
|
|
await pool.call("box", protocol.READ, {})
|
|
|
|
|
|
|
|
|
|
assert not client.alive and "box" not in pool._nodes
|
|
|
|
|
await pool.close_all()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
|
|
|
("returncode", "stderr", "said"),
|
|
|
|
|
[
|
|
|
|
|
(255, "", "machine 'Lab box' (box) could not be reached"),
|
|
|
|
|
(
|
|
|
|
|
127,
|
|
|
|
|
"",
|
|
|
|
|
"this Raven's node is not installed on machine 'Lab box' (box); "
|
|
|
|
|
"install it with `raven ops connection install-node box`",
|
|
|
|
|
),
|
|
|
|
|
(
|
|
|
|
|
126,
|
|
|
|
|
"sh: exec: /root/.raven-node/x/bin/python: cannot execute: No such file or directory",
|
|
|
|
|
"this Raven's node is not installed on machine 'Lab box' (box); "
|
|
|
|
|
"install it with `raven ops connection install-node box`",
|
|
|
|
|
),
|
|
|
|
|
(
|
|
|
|
|
1,
|
|
|
|
|
"bash: no job control in this shell\nTraceback (most recent call last):\nMemoryError: out of memory\n",
|
|
|
|
|
"the Raven node on machine 'Lab box' (box) stopped: MemoryError: out of memory",
|
|
|
|
|
),
|
|
|
|
|
(None, None, "the Raven node on machine 'Lab box' (box) stopped"),
|
|
|
|
|
],
|
|
|
|
|
)
|
|
|
|
|
async def test_a_connection_that_ended_mid_call_is_one_sentence(tmp_path, returncode, stderr, said):
|
|
|
|
|
pool = NodePool(backoff_s=())
|
|
|
|
|
client = _live(pool, tmp_path, raises=AcpConnectionError("gone", stderr=stderr, returncode=returncode))
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError) as caught:
|
|
|
|
|
await pool.call("box", protocol.READ, {})
|
|
|
|
|
|
|
|
|
|
assert str(caught.value) == said
|
|
|
|
|
_no_address(str(caught.value))
|
|
|
|
|
assert not client.alive and "box" not in pool._nodes
|
|
|
|
|
await pool.close_all()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_node_left_idle_is_closed_when_another_machine_is_asked_for_unless_an_agent_runs_on_it(tmp_path):
|
|
|
|
|
pool = NodePool(idle_close_s=60.0)
|
|
|
|
|
old = _live(pool, tmp_path, "old")
|
|
|
|
|
busy = _live(pool, tmp_path, "busy")
|
|
|
|
|
recent = _live(pool, tmp_path, "recent")
|
|
|
|
|
asked = _live(pool, tmp_path, "asked")
|
|
|
|
|
for machine_id in ("old", "busy", "asked"):
|
|
|
|
|
pool._nodes[machine_id].used = time.monotonic() - 61.0
|
|
|
|
|
pool._nodes["busy"].channels[1] = ChannelEndpoint(busy, 1, target=_box("busy"), agent="a") # type: ignore[arg-type]
|
|
|
|
|
|
|
|
|
|
await pool.call("asked", protocol.READ, {})
|
|
|
|
|
|
|
|
|
|
assert not old.alive and busy.alive and recent.alive and asked.alive
|
|
|
|
|
assert sorted(pool.live()) == ["asked", "busy", "recent"]
|
|
|
|
|
assert old.asked == [], "a link closed by choice is just closed; its agents stay with the machine"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_call_on_its_way_to_a_node_keeps_its_link_however_long_ago_it_was_last_used(tmp_path):
|
|
|
|
|
pool = NodePool(idle_close_s=60.0)
|
|
|
|
|
|
|
|
|
|
class _Slow(_Client):
|
|
|
|
|
async def request(self, method: str, params: dict[str, Any], *, timeout: float) -> Any:
|
|
|
|
|
await asyncio.sleep(0.3)
|
|
|
|
|
return await super().request(method, params, timeout=timeout)
|
|
|
|
|
|
|
|
|
|
_live(pool, tmp_path, "slow")
|
|
|
|
|
pool._nodes["slow"].client = slow = _Slow() # type: ignore[assignment]
|
|
|
|
|
pool._nodes["slow"].used = time.monotonic() - 61.0
|
|
|
|
|
_live(pool, tmp_path, "other")
|
|
|
|
|
on_its_way = asyncio.create_task(pool.call("slow", protocol.READ, {}))
|
|
|
|
|
await asyncio.sleep(0.05)
|
|
|
|
|
|
|
|
|
|
await pool.call("other", protocol.READ, {})
|
|
|
|
|
|
|
|
|
|
assert slow.alive and "slow" in pool.live(), "a link with a call on its way is in use, not idle"
|
|
|
|
|
assert await on_its_way == {"text": "ok"}
|
|
|
|
|
assert pool._nodes["slow"].calls == 0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_live_lists_only_the_nodes_still_running_and_close_ends_them(tmp_path):
|
|
|
|
|
pool = NodePool()
|
|
|
|
|
one = _live(pool, tmp_path, "one")
|
|
|
|
|
two = _live(pool, tmp_path, "two")
|
|
|
|
|
two.alive = False
|
|
|
|
|
assert pool.live() == ["one"]
|
|
|
|
|
|
|
|
|
|
await pool.close("one")
|
|
|
|
|
assert not one.alive and pool.live() == []
|
|
|
|
|
|
|
|
|
|
three = _live(pool, tmp_path, "three")
|
|
|
|
|
await pool.close_all()
|
|
|
|
|
assert not three.alive and pool._nodes == {}
|
|
|
|
|
await pool.close("never-started")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_one_pool_serves_the_process_until_it_is_closed(tmp_path, monkeypatch):
|
|
|
|
|
monkeypatch.setattr(node_client, "_POOL", None)
|
|
|
|
|
pool = node_client.get_node_pool()
|
|
|
|
|
assert node_client.get_node_pool() is pool
|
|
|
|
|
client = _live(pool, tmp_path)
|
|
|
|
|
|
|
|
|
|
await node_client.close_node_pool()
|
|
|
|
|
|
|
|
|
|
assert not client.alive
|
|
|
|
|
assert node_client.get_node_pool() is not pool
|
|
|
|
|
await node_client.close_node_pool()
|
|
|
|
|
await node_client.close_node_pool()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- the heartbeat, with the connection stood in for ------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class _Deaf(_Client):
|
|
|
|
|
"""A connection that answers everything but a ping."""
|
|
|
|
|
|
|
|
|
|
async def request(self, method: str, params: dict[str, Any], *, timeout: float) -> Any:
|
|
|
|
|
self.asked.append((method, params, timeout))
|
|
|
|
|
if method == protocol.PING:
|
|
|
|
|
raise AcpTimeoutError("no answer", method=method)
|
|
|
|
|
return self.result
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_heartbeat_unanswered_loses_the_link_tells_its_agents_and_reconnects(tmp_path, monkeypatch):
|
|
|
|
|
pool = NodePool(heartbeat_s=0.01, lost_after_s=0.5, backoff_s=(0.01, 0.01))
|
|
|
|
|
client = _Deaf()
|
|
|
|
|
log = tmp_path / "box.log"
|
|
|
|
|
log.write_text("", encoding="utf-8")
|
|
|
|
|
node = _Node(client, _box(), log, {"home": "/h"}, {}) # type: ignore[arg-type]
|
|
|
|
|
pool._nodes["box"] = node
|
|
|
|
|
endpoint = ChannelEndpoint(client, 1, target=_box(), agent="echo@box") # type: ignore[arg-type]
|
|
|
|
|
node.channels[1] = endpoint
|
|
|
|
|
tried: list[str] = []
|
|
|
|
|
|
|
|
|
|
async def launch(machine_id: str) -> _Node:
|
|
|
|
|
tried.append(machine_id)
|
|
|
|
|
if len(tried) == 1:
|
|
|
|
|
raise NodeError("machine 'Lab box' (box) could not be reached")
|
|
|
|
|
return _Node(_Client(), _box(), log, {"home": "/h"}, {}) # type: ignore[arg-type]
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(pool, "_launch", launch)
|
|
|
|
|
node.heartbeat = asyncio.create_task(pool._heartbeat("box", node))
|
|
|
|
|
|
|
|
|
|
await _until(lambda: not client.alive)
|
|
|
|
|
assert endpoint.returncode is None and endpoint.ended() is None, "the agent is not ended: its link is"
|
|
|
|
|
assert endpoint.detached == "machine 'Lab box' (box) did not answer a heartbeat within 0s"
|
|
|
|
|
status = pool.status("box")
|
|
|
|
|
assert status.state == "lost" and status.agents == 1
|
|
|
|
|
assert status.said() == (
|
|
|
|
|
"link lost: machine 'Lab box' (box) did not answer a heartbeat within 0s; "
|
|
|
|
|
"1 task continues on the machine; reconnecting"
|
|
|
|
|
)
|
|
|
|
|
await _until(lambda: len(tried) == 2 and pool.live() == ["box"] and endpoint.detached is None)
|
|
|
|
|
again = pool._nodes["box"].client
|
|
|
|
|
# Among the new link's requests, not its last: a heartbeat may well follow the attach.
|
fix(node): an attach replays the agent's stderr past the line it last read (#977)
## Summary
Follow-up to LivXue's review note on #957 (the pick-up's binding at
`cli_agent.py:1393`). The machine's daemon kept an agent's stderr in the
task's record (`stderr.log`) but sent it only to the connection holding
the
task when a line was written, and `agent/attach` replayed `frames.jsonl`
alone. So what an agent said only on stderr while no connection held it
never
reached the host:
- **The session id a CLI names only on stderr** (hermes does). The
pick-up
#957 added binds a create's handle to the id in its output; for such a
CLI
it found none, warned "not resumable", and the next turn to the instance
opened a new session.
- **The reason a command failed.** A run picked up after this computer
stopped was reported as `exited 3` where the dispatch would have said
`exited 3: token expired`. This part predates #957.
- **The stderr written while a link was down**, on a run still being
watched, the same way.
The fix is in the node and the host's link to it (`raven/node/`):
- The daemon numbers stderr lines as it numbers frames (`agent/stderr`
carries `n`), and `agent/attach` takes an optional `stderr_after`: the
stderr lines past that one are replayed after the frames, under the lock
that forwards a new line, so none is sent twice or missed at the
handover.
It works the same for a live task, a finished one, and a record a daemon
that never ran the task reads. An attach that does not ask hears no
stderr,
as before; a `stderr_after` that is not a whole number is refused like a
bad `after`.
- The host's `ChannelEndpoint` counts the stderr lines it got
(`stderr_received`). A cli pick-up (`NodePool.attach`, a process that
read
none of them) asks from the first line; an acp pick-up from the line the
run's record kept as the prompt went out (`RemoteRun.stderr_received`,
follow-up commit below); a re-attach after a lost link
(`NodePool._reattach`) asks from the last one it read.
The cli dispatch and pick-up read stderr off the endpoint as before and
now
get all of it, so the reason and the session id are there. An acp
pick-up's
connection drains the same endpoint into the stderr tail it quotes in
its
errors and into its journal, so what the agent wrote during the turn
while
this computer was away reaches them too; the acp run record keeps the
stderr
line count as the prompt goes out, so the earlier prompts' lines are not
read again (follow-up commit below). No protocol bump:
the new fields are optional, and a node runs exactly the host's code
anyway
(protocol and digest are checked at every connection).
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `tests/test_node_stderr_replay.py`, new, 22 tests: the daemon numbers
and
keeps stderr lines; an attach replays exactly the lines past
`stderr_after`
(from a live task, a finished one, and a record a daemon that never ran
it
reads), none when not asked, none from a record with no `stderr.log`,
and
refuses a bad `stderr_after`; the endpoint counts by the node's numbers;
a
re-attach and a pick-up ask from the right line. End to end on the
stand-in machine (real daemon, stand-in ssh): a left cli run's reason on
stderr reaches its pick-up (said before and after this computer
stopped),
a left create that names its session only on stderr is bound to it by
the
pick-up (both orders), and a reason written while the link was down
reaches the run watching it. 18 fail on main (2c2718122); the 4 that
pass
there pin what does not change (no `stderr_after`, past the last line,
past the end, no `stderr.log`).
- Changed assertions: the four attach-parameter assertions in
`tests/test_node_client.py` and the forwarded-stderr one in
`tests/test_node_server.py` include the new fields.
- Full suite: 32065 passed, 173 skipped, 2 xfailed, 3 failed -- the same
3
that fail on main (2c2718122) on this computer when run alone there: a
host
without LibreOffice, and two simulation cases.
- Diff coverage: 100.00% (30/30 executable changed lines; `coverage
json`, then
`scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`).
- `ruff check`, `ruff format --check`, `ty check`, `lint-imports`,
pre-commit on the diff; `scripts/check_commit_messages.py`,
`scripts/check_source_language.py` and `scripts/check_large_files.py` on
`origin/main..HEAD`: clean.
- Docs: CONTEXT.md's Raven node entry, the protocol module's notes, and
the
machines guide's "Disconnects and restarts" section in English and
Chinese
now say the daemon keeps and replays stderr; the changelog has an entry
under Fixed. `tests/test_docs_site.py` and `tests/test_living_docs.py`
pass
(18), `mkdocs build --strict` passes.
- Real machine, ravenx2, an isolated node directory per tree. The host
process that dispatched was killed with SIGKILL and the run picked up by
a
new process; the link was dropped by killing the local ssh. The command
writes `token expired` (or `session slow-A1`) on stderr:
| Probe | main (2c2718122) | this branch |
|---|---|---|
| reason written after the host process was killed | `exited 3` |
`exited 3: token expired` |
| reason written before it was killed | `exited 3` | `exited 3: token
expired` |
| reason written while the link was down | `exited 3` | `exited 3: token
expired` |
| create left before its commit, session id only on stderr | "not
resumable", nothing bound | bound to `slow-A1`, no warning |
The machine was cleaned after: the probe's two daemons ended by pid, its
directory removed.
- The runs above are on main 2c2718122 (#957 merged). The branch was
then
rebased onto 400b30dfa, whose two new commits touch only TUI copy and
i18n; the node, docs and range checks were run again there (238 passed,
1
skipped). It merges cleanly with the open #964 and #965 (`git
merge-tree`).
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
- Follow-up commit e0468fa88, after the agent review's F1 (an acp
pick-up asked
for the stderr from the agent's first line, across every earlier prompt
of
its process): `RemoteRun.stderr_received`, taken in
`Tracker.prompt_sent`
off the endpoint's count, and passed through `NodePool.attach(...,
stderr_after=...)` by the pick-up; a cli pick-up keeps asking from 0; a
record from before the field reads with 0. Regression on the stand-in
machine
(`test_a_pick_up_reads_the_turn_s_own_stderr_and_not_what_the_prompts_before_it_wrote`):
two turns of 150 stderr lines, a third interrupted after its prompt, the
next start's pick-up -- before the change its tail held 200 lines, 50 of
them the second turn's; after it, the third turn's 150 and nothing else.
Unit tests pin the record, the pick-up's attach arguments and the
default
of 0, and the tracker's wiring. The related files 259 passed; full suite
32067 passed with the three failures this Mac always has, plus one test
whose stand-in `Tracker` lacked the new keyword, fixed in the same
commit.
Diff coverage of the changed lines: see CI.
## Risk
The replayed lines are the agent's own stderr, already kept in the
task's
owner-only record and already sent live to whichever connection held the
task; nothing new leaves the machine. `stderr_after` and `n` are
optional: an
attach without `stderr_after` behaves as before, and a line without a
number
counts on from the last. A cli pick-up replays the whole `stderr.log`,
as it
already replays every frame of the command; an acp pick-up replays the
lines
past the count its record kept as the prompt went out, so the replay is
bounded to the turn, and the connection keeps a bounded tail of it.
Rollback
is reverting the commit.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follow-up to #957, for this review note:
https://github.com/EverMind-AI/Raven/pull/957#discussion_r4282448689
---------
Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-17 21:08:32 +08:00
|
|
|
attach = (protocol.AGENT_ATTACH, {"channel": 1, "task": endpoint.task, "after": 0, "stderr_after": 0})
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
assert attach in [asked[:2] for asked in again.asked]
|
|
|
|
|
assert pool.status("box").state == "connected" and pool._nodes["box"].channels[1] is endpoint
|
|
|
|
|
await pool.close_all()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_lost_link_that_does_not_come_back_keeps_its_last_sentence(tmp_path, monkeypatch):
|
|
|
|
|
pool = NodePool(heartbeat_s=0.01, lost_after_s=0.5, backoff_s=(0.01,))
|
|
|
|
|
client = _Deaf()
|
|
|
|
|
node = _Node(client, _box(), tmp_path / "box.log", {}, {}) # type: ignore[arg-type]
|
|
|
|
|
pool._nodes["box"] = node
|
|
|
|
|
|
|
|
|
|
async def launch(machine_id: str) -> _Node:
|
|
|
|
|
raise NodeError("machine 'Lab box' (box) refused the login; check the key it is registered with")
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(pool, "_launch", launch)
|
|
|
|
|
node.heartbeat = asyncio.create_task(pool._heartbeat("box", node))
|
|
|
|
|
|
|
|
|
|
await _until(lambda: not client.alive and "box" not in pool._reconnects)
|
|
|
|
|
assert pool.status("box") == node_client.MachineStatus(
|
|
|
|
|
"error", "machine 'Lab box' (box) refused the login; check the key it is registered with"
|
|
|
|
|
)
|
|
|
|
|
assert pool.status("elsewhere") == node_client.MachineStatus("off") and pool.status("elsewhere").said() == ""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_channel_endpoint_carries_frames_both_ways_and_ends_with_the_agent(tmp_path):
|
|
|
|
|
link = _Client()
|
|
|
|
|
endpoint = ChannelEndpoint(link, 3, target=_box(), agent="echo@box") # type: ignore[arg-type]
|
|
|
|
|
|
|
|
|
|
await endpoint.write(b'{"jsonrpc":"2.0","id":1,"method":"x"}\n')
|
|
|
|
|
endpoint.feed_frame('{"jsonrpc":"2.0","id":1,"result":{}}')
|
|
|
|
|
endpoint.feed_stderr("agent: hello")
|
|
|
|
|
endpoint.exited(3)
|
|
|
|
|
endpoint.feed_frame("too late")
|
|
|
|
|
|
|
|
|
|
assert link.said == [(protocol.AGENT_FRAME, {"channel": 3, "line": '{"jsonrpc":"2.0","id":1,"method":"x"}'})]
|
feat(*): read, write and deliver files on a registered machine by address (#922)
## Summary
Third of four changes that let a Raven dispatch work to the machines in
its machine registry. #920 is merged; this branch is opened against main
ahead of #921, so the diff here includes #921's commits as well: review
from "read, write and deliver files on a registered machine by address"
on (the commits before it are #921's, carried here until it merges).
What this does:
- The file tools take a path on a registered machine as it is written
for scp -- `user@host:/path`, or `scp://user@host:port//path` when the
port is not 22 -- and answer it through the machine's daemon
(`raven/agent/tools/remote_files.py`, hooked from `filesystem.py` and
`file_search.py`): `read_file`, `list_dir`, `find` and `grep` run
Raven's own tools there and bring back the answer, so a read of fifty
lines of a large file moves those fifty lines; `write_file` and
`edit_file` write there, with the digest of that agent's own last read
-- the conversation's, or its sub-agent run's -- as the expected
state, so a file changed since it read it is refused and nothing is
written, and another agent's read does not stand in for its own. The
path is resolved against the registry
only, never this computer's ssh config, so one path names one machine
on any computer; one that matches no row is refused, not guessed.
- What is shown back as an address is only what the tool says about
paths -- its errors, its headers and footers, a write's account, a
diff's two header lines; a read's numbered lines, a search's matches
and context, and a diff's body come back as they are, so the model
edits against the bytes the file holds. The shell's refusal of typed
`ssh` to a registered machine (`raven/agent/tools/shell.py`,
`machine_exec.py`) reads the program in any spelling -- a path, a
backslash, and on a Windows host `ssh.exe`, `SSH.EXE` or a whole
`C:\...\ssh.exe`, with the backslashes the shell there keeps and
without the quotes it takes off (a quoted piece and a bare one next to
it making one word), so a quoted destination or port is matched like a
bare one -- and, with the lane on, names the file tools. `scp`, `sftp`
and `rsync` to a registered machine stay what they are on main: a copy
starts nothing there and passes exec's own question like any command,
and the file tools cannot carry a directory, a binary file or a large
one, so the copy stays the route for those. A drive-relative or
extended Windows path (`C:file`, `\\?\C:\...`, UNC) is this
computer's, not a machine nobody registered.
- Permissions (`raven/permissions/`): a path inside the directories
Raven's agents work in on that machine -- its session directory, its
`remoteCwd`, the row's listed `paths` -- is read without asking;
elsewhere a read asks once per machine and directory for the session.
A write there asks wherever it falls, as a write here does, and its
grant is for the one file, apart from any read. The daemon checks the
roots again on the machine, links included.
- `deliver_files` fetches one file by its address, whole and checked,
under a 20 MB cap, into `<workspace>/pulled/<machine>/<key>/<name>`,
the key a digest of the file's path on the machine, so two paths never
share a landing: not two files of one name from two directories, and
not two names this computer's filesystem reads as one (`Report.txt`
and `report.txt`, a trailing dot). The name beside the key is written
as one this computer keeps (a backslash, what Windows reserves in a
name, control characters and the percent sign itself as
percent-codes), and a landing that would not resolve under the
machine's delivery directory is refused before anything is made; a
bigger file is named with its size and read there instead. Only a file
under the directories this Raven's agents work in
on that machine is fetched: the tool asks nothing, and a read some
conversation was allowed is no evidence for this one, so a file
elsewhere is refused and read there instead. Nothing reclaims
`pulled/`: a delivery is opened from its landing for as long as its
record lasts, and a bound by size or age is left out of this change on
purpose.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_remote_file_tools.py
tests/test_permissions_remote_read.py tests/test_deliver_files_tool.py
tests/test_shell_machine_channel.py tests/test_ops_remote_path.py
tests/test_permissions_gate.py tests/test_import_cycle_budget.py
tests/test_node_server.py tests/test_node_client.py
tests/test_subagent_acp_remote.py tests/test_acp_remote.py
tests/test_cli_ops_connection_commands.py`
-- all passed on this tree (703 in the run that covered the first
eight groups, the node's own suite and the real-node cases among them).
Covered: each tool on a remote path answers as its local tool would,
through a stood-in link and a real daemon; a path outside the roots is
refused on both sides; a write with a stale digest is refused and
writes nothing; the address form on what the tool says about paths,
and a read's lines, a grep's matches and a diff's body untouched; the
shell's refusal of typed ssh to a registered machine, by host, by
alias, in a pipeline, as `ssh.exe` and `SSH.EXE`, and with a whole
Windows path or a quoted destination or port when the host is Windows,
a copy with scp, sftp or rsync not refused, and the file tools named
only with the lane on; a fetched name holding the
other platform's separators lands as one name under the machine's
directory and nothing lands elsewhere, and a landing that would
resolve outside fetches nothing; a Windows path form read here with the
lane on; the read
permission inside the roots and the question elsewhere, remembered per
machine and directory, a write asked wherever it falls and its grant
kept per file and apart from a read; a read's digest kept per agent,
so another agent's read lets no stale overwrite through and an agent
that never read the file expects nothing; a delivery fetched whole
under a key of its path (two files of one name, two names a
filesystem reads as one, and the percent-code pair all land apart),
refused over the cap and refused outside the roots whatever was read
before.
- `uv run pytest` on this tree after the merge with main -- 31210
passed,
231 skipped, 2 xfailed, 4 failed, every one of the four in a test that
needs what this machine lacks (`test_rpc_files.py`, the two simulation
cases and the kimi-code case that fail the same way on main here); the
groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Diff coverage of this change against main after the merge: 95.27% (805
of
845 executable changed lines), against the 90% gate. The node suite,
the remote-path tests and the real-node cases among the groups above run
on the merged tree.
- Real machines (two registered Linux machines): fifty lines from the
middle of a 5 MB file read through the daemon in 30 to 35 ms, byte for
byte what this computer reads; list, find and grep answered; a write
and an edit landed as the local tools would, and a write after the file
changed was refused; a path outside the roots refused; a file written
by an agent there fetched back by address. The shell's refusals are
covered by the tests above.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint`,
`ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
User docs follow once the lane is no longer debug-only.
## Risk
Behind the debug lane, as the two changes before it. With it on, a
remote
path reaches the machine only through its daemon, inside the roots each
call names and under the read and write questions above; the shell
cannot
run a command on a registered machine by address, and a copy to or from
it
with scp, sftp or rsync is the plain shell's behind exec's own question,
as
on main. A write refused for a stale
digest writes nothing. A delivery fetches at most 20 MB per file into
the
workspace, and only from the directories the agents work in on the
machine. Rollback is a plain revert.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#920 (merged); depends on #921.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 16:10:18 +08:00
|
|
|
assert await endpoint.readline() == b'{"jsonrpc":"2.0","id":1,"result":{}}\n'
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
assert await endpoint.readline() == b""
|
feat(*): read, write and deliver files on a registered machine by address (#922)
## Summary
Third of four changes that let a Raven dispatch work to the machines in
its machine registry. #920 is merged; this branch is opened against main
ahead of #921, so the diff here includes #921's commits as well: review
from "read, write and deliver files on a registered machine by address"
on (the commits before it are #921's, carried here until it merges).
What this does:
- The file tools take a path on a registered machine as it is written
for scp -- `user@host:/path`, or `scp://user@host:port//path` when the
port is not 22 -- and answer it through the machine's daemon
(`raven/agent/tools/remote_files.py`, hooked from `filesystem.py` and
`file_search.py`): `read_file`, `list_dir`, `find` and `grep` run
Raven's own tools there and bring back the answer, so a read of fifty
lines of a large file moves those fifty lines; `write_file` and
`edit_file` write there, with the digest of that agent's own last read
-- the conversation's, or its sub-agent run's -- as the expected
state, so a file changed since it read it is refused and nothing is
written, and another agent's read does not stand in for its own. The
path is resolved against the registry
only, never this computer's ssh config, so one path names one machine
on any computer; one that matches no row is refused, not guessed.
- What is shown back as an address is only what the tool says about
paths -- its errors, its headers and footers, a write's account, a
diff's two header lines; a read's numbered lines, a search's matches
and context, and a diff's body come back as they are, so the model
edits against the bytes the file holds. The shell's refusal of typed
`ssh` to a registered machine (`raven/agent/tools/shell.py`,
`machine_exec.py`) reads the program in any spelling -- a path, a
backslash, and on a Windows host `ssh.exe`, `SSH.EXE` or a whole
`C:\...\ssh.exe`, with the backslashes the shell there keeps and
without the quotes it takes off (a quoted piece and a bare one next to
it making one word), so a quoted destination or port is matched like a
bare one -- and, with the lane on, names the file tools. `scp`, `sftp`
and `rsync` to a registered machine stay what they are on main: a copy
starts nothing there and passes exec's own question like any command,
and the file tools cannot carry a directory, a binary file or a large
one, so the copy stays the route for those. A drive-relative or
extended Windows path (`C:file`, `\\?\C:\...`, UNC) is this
computer's, not a machine nobody registered.
- Permissions (`raven/permissions/`): a path inside the directories
Raven's agents work in on that machine -- its session directory, its
`remoteCwd`, the row's listed `paths` -- is read without asking;
elsewhere a read asks once per machine and directory for the session.
A write there asks wherever it falls, as a write here does, and its
grant is for the one file, apart from any read. The daemon checks the
roots again on the machine, links included.
- `deliver_files` fetches one file by its address, whole and checked,
under a 20 MB cap, into `<workspace>/pulled/<machine>/<key>/<name>`,
the key a digest of the file's path on the machine, so two paths never
share a landing: not two files of one name from two directories, and
not two names this computer's filesystem reads as one (`Report.txt`
and `report.txt`, a trailing dot). The name beside the key is written
as one this computer keeps (a backslash, what Windows reserves in a
name, control characters and the percent sign itself as
percent-codes), and a landing that would not resolve under the
machine's delivery directory is refused before anything is made; a
bigger file is named with its size and read there instead. Only a file
under the directories this Raven's agents work in
on that machine is fetched: the tool asks nothing, and a read some
conversation was allowed is no evidence for this one, so a file
elsewhere is refused and read there instead. Nothing reclaims
`pulled/`: a delivery is opened from its landing for as long as its
record lasts, and a bound by size or age is left out of this change on
purpose.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_remote_file_tools.py
tests/test_permissions_remote_read.py tests/test_deliver_files_tool.py
tests/test_shell_machine_channel.py tests/test_ops_remote_path.py
tests/test_permissions_gate.py tests/test_import_cycle_budget.py
tests/test_node_server.py tests/test_node_client.py
tests/test_subagent_acp_remote.py tests/test_acp_remote.py
tests/test_cli_ops_connection_commands.py`
-- all passed on this tree (703 in the run that covered the first
eight groups, the node's own suite and the real-node cases among them).
Covered: each tool on a remote path answers as its local tool would,
through a stood-in link and a real daemon; a path outside the roots is
refused on both sides; a write with a stale digest is refused and
writes nothing; the address form on what the tool says about paths,
and a read's lines, a grep's matches and a diff's body untouched; the
shell's refusal of typed ssh to a registered machine, by host, by
alias, in a pipeline, as `ssh.exe` and `SSH.EXE`, and with a whole
Windows path or a quoted destination or port when the host is Windows,
a copy with scp, sftp or rsync not refused, and the file tools named
only with the lane on; a fetched name holding the
other platform's separators lands as one name under the machine's
directory and nothing lands elsewhere, and a landing that would
resolve outside fetches nothing; a Windows path form read here with the
lane on; the read
permission inside the roots and the question elsewhere, remembered per
machine and directory, a write asked wherever it falls and its grant
kept per file and apart from a read; a read's digest kept per agent,
so another agent's read lets no stale overwrite through and an agent
that never read the file expects nothing; a delivery fetched whole
under a key of its path (two files of one name, two names a
filesystem reads as one, and the percent-code pair all land apart),
refused over the cap and refused outside the roots whatever was read
before.
- `uv run pytest` on this tree after the merge with main -- 31210
passed,
231 skipped, 2 xfailed, 4 failed, every one of the four in a test that
needs what this machine lacks (`test_rpc_files.py`, the two simulation
cases and the kimi-code case that fail the same way on main here); the
groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Diff coverage of this change against main after the merge: 95.27% (805
of
845 executable changed lines), against the 90% gate. The node suite,
the remote-path tests and the real-node cases among the groups above run
on the merged tree.
- Real machines (two registered Linux machines): fifty lines from the
middle of a 5 MB file read through the daemon in 30 to 35 ms, byte for
byte what this computer reads; list, find and grep answered; a write
and an edit landed as the local tools would, and a write after the file
changed was refused; a path outside the roots refused; a file written
by an agent there fetched back by address. The shell's refusals are
covered by the tests above.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint`,
`ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
User docs follow once the lane is no longer debug-only.
## Risk
Behind the debug lane, as the two changes before it. With it on, a
remote
path reaches the machine only through its daemon, inside the roots each
call names and under the read and write questions above; the shell
cannot
run a command on a registered machine by address, and a copy to or from
it
with scp, sftp or rsync is the plain shell's behind exec's own question,
as
on main. A write refused for a stale
digest writes nothing. A delivery fetches at most 20 MB per file into
the
workspace, and only from the directories the agents work in on the
machine. Rollback is a plain revert.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#920 (merged); depends on #921.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 16:10:18 +08:00
|
|
|
assert await endpoint.readerr() == b"agent: hello\n" and await endpoint.readerr() == b""
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
assert endpoint.returncode == 3 and await endpoint.wait() == 3 and endpoint.ended() is None
|
|
|
|
|
with pytest.raises(BrokenPipeError):
|
|
|
|
|
await endpoint.write(b"{}\n")
|
|
|
|
|
await endpoint.end()
|
|
|
|
|
assert all(method != protocol.AGENT_CANCEL for method, *_ in link.asked), "nothing to cancel once it has exited"
|
|
|
|
|
|
|
|
|
|
|
feat(*): read, write and deliver files on a registered machine by address (#922)
## Summary
Third of four changes that let a Raven dispatch work to the machines in
its machine registry. #920 is merged; this branch is opened against main
ahead of #921, so the diff here includes #921's commits as well: review
from "read, write and deliver files on a registered machine by address"
on (the commits before it are #921's, carried here until it merges).
What this does:
- The file tools take a path on a registered machine as it is written
for scp -- `user@host:/path`, or `scp://user@host:port//path` when the
port is not 22 -- and answer it through the machine's daemon
(`raven/agent/tools/remote_files.py`, hooked from `filesystem.py` and
`file_search.py`): `read_file`, `list_dir`, `find` and `grep` run
Raven's own tools there and bring back the answer, so a read of fifty
lines of a large file moves those fifty lines; `write_file` and
`edit_file` write there, with the digest of that agent's own last read
-- the conversation's, or its sub-agent run's -- as the expected
state, so a file changed since it read it is refused and nothing is
written, and another agent's read does not stand in for its own. The
path is resolved against the registry
only, never this computer's ssh config, so one path names one machine
on any computer; one that matches no row is refused, not guessed.
- What is shown back as an address is only what the tool says about
paths -- its errors, its headers and footers, a write's account, a
diff's two header lines; a read's numbered lines, a search's matches
and context, and a diff's body come back as they are, so the model
edits against the bytes the file holds. The shell's refusal of typed
`ssh` to a registered machine (`raven/agent/tools/shell.py`,
`machine_exec.py`) reads the program in any spelling -- a path, a
backslash, and on a Windows host `ssh.exe`, `SSH.EXE` or a whole
`C:\...\ssh.exe`, with the backslashes the shell there keeps and
without the quotes it takes off (a quoted piece and a bare one next to
it making one word), so a quoted destination or port is matched like a
bare one -- and, with the lane on, names the file tools. `scp`, `sftp`
and `rsync` to a registered machine stay what they are on main: a copy
starts nothing there and passes exec's own question like any command,
and the file tools cannot carry a directory, a binary file or a large
one, so the copy stays the route for those. A drive-relative or
extended Windows path (`C:file`, `\\?\C:\...`, UNC) is this
computer's, not a machine nobody registered.
- Permissions (`raven/permissions/`): a path inside the directories
Raven's agents work in on that machine -- its session directory, its
`remoteCwd`, the row's listed `paths` -- is read without asking;
elsewhere a read asks once per machine and directory for the session.
A write there asks wherever it falls, as a write here does, and its
grant is for the one file, apart from any read. The daemon checks the
roots again on the machine, links included.
- `deliver_files` fetches one file by its address, whole and checked,
under a 20 MB cap, into `<workspace>/pulled/<machine>/<key>/<name>`,
the key a digest of the file's path on the machine, so two paths never
share a landing: not two files of one name from two directories, and
not two names this computer's filesystem reads as one (`Report.txt`
and `report.txt`, a trailing dot). The name beside the key is written
as one this computer keeps (a backslash, what Windows reserves in a
name, control characters and the percent sign itself as
percent-codes), and a landing that would not resolve under the
machine's delivery directory is refused before anything is made; a
bigger file is named with its size and read there instead. Only a file
under the directories this Raven's agents work in
on that machine is fetched: the tool asks nothing, and a read some
conversation was allowed is no evidence for this one, so a file
elsewhere is refused and read there instead. Nothing reclaims
`pulled/`: a delivery is opened from its landing for as long as its
record lasts, and a bound by size or age is left out of this change on
purpose.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_remote_file_tools.py
tests/test_permissions_remote_read.py tests/test_deliver_files_tool.py
tests/test_shell_machine_channel.py tests/test_ops_remote_path.py
tests/test_permissions_gate.py tests/test_import_cycle_budget.py
tests/test_node_server.py tests/test_node_client.py
tests/test_subagent_acp_remote.py tests/test_acp_remote.py
tests/test_cli_ops_connection_commands.py`
-- all passed on this tree (703 in the run that covered the first
eight groups, the node's own suite and the real-node cases among them).
Covered: each tool on a remote path answers as its local tool would,
through a stood-in link and a real daemon; a path outside the roots is
refused on both sides; a write with a stale digest is refused and
writes nothing; the address form on what the tool says about paths,
and a read's lines, a grep's matches and a diff's body untouched; the
shell's refusal of typed ssh to a registered machine, by host, by
alias, in a pipeline, as `ssh.exe` and `SSH.EXE`, and with a whole
Windows path or a quoted destination or port when the host is Windows,
a copy with scp, sftp or rsync not refused, and the file tools named
only with the lane on; a fetched name holding the
other platform's separators lands as one name under the machine's
directory and nothing lands elsewhere, and a landing that would
resolve outside fetches nothing; a Windows path form read here with the
lane on; the read
permission inside the roots and the question elsewhere, remembered per
machine and directory, a write asked wherever it falls and its grant
kept per file and apart from a read; a read's digest kept per agent,
so another agent's read lets no stale overwrite through and an agent
that never read the file expects nothing; a delivery fetched whole
under a key of its path (two files of one name, two names a
filesystem reads as one, and the percent-code pair all land apart),
refused over the cap and refused outside the roots whatever was read
before.
- `uv run pytest` on this tree after the merge with main -- 31210
passed,
231 skipped, 2 xfailed, 4 failed, every one of the four in a test that
needs what this machine lacks (`test_rpc_files.py`, the two simulation
cases and the kimi-code case that fail the same way on main here); the
groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Diff coverage of this change against main after the merge: 95.27% (805
of
845 executable changed lines), against the 90% gate. The node suite,
the remote-path tests and the real-node cases among the groups above run
on the merged tree.
- Real machines (two registered Linux machines): fifty lines from the
middle of a 5 MB file read through the daemon in 30 to 35 ms, byte for
byte what this computer reads; list, find and grep answered; a write
and an edit landed as the local tools would, and a write after the file
changed was refused; a path outside the roots refused; a file written
by an agent there fetched back by address. The shell's refusals are
covered by the tests above.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint`,
`ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
User docs follow once the lane is no longer debug-only.
## Risk
Behind the debug lane, as the two changes before it. With it on, a
remote
path reaches the machine only through its daemon, inside the roots each
call names and under the read and write questions above; the shell
cannot
run a command on a registered machine by address, and a copy to or from
it
with scp, sftp or rsync is the plain shell's behind exec's own question,
as
on main. A write refused for a stale
digest writes nothing. A delivery fetches at most 20 MB per file into
the
workspace, and only from the directories the agents work in on the
machine. Rollback is a plain revert.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#920 (merged); depends on #921.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 16:10:18 +08:00
|
|
|
async def test_a_blank_line_is_a_frame_the_endpoint_hands_on_as_a_pipe_would():
|
|
|
|
|
endpoint = ChannelEndpoint(_Client(), 1, target=_box(), agent="cli@box") # type: ignore[arg-type]
|
|
|
|
|
endpoint.feed_frame("a", 1)
|
|
|
|
|
endpoint.feed_frame("", 2)
|
|
|
|
|
endpoint.feed_frame("b", 3)
|
|
|
|
|
endpoint.exited(0)
|
|
|
|
|
|
|
|
|
|
assert [await endpoint.readline() for _ in range(4)] == [b"a\n", b"\n", b"b\n", b""]
|
|
|
|
|
assert endpoint.received == 3, "the blank line counts, so an attach picks up where this one stopped"
|
|
|
|
|
|
|
|
|
|
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
async def test_ending_a_running_channel_is_a_cancel_on_the_link(tmp_path):
|
|
|
|
|
link = _Client()
|
|
|
|
|
endpoint = ChannelEndpoint(link, 2, target=_box(), agent="echo@box") # type: ignore[arg-type]
|
|
|
|
|
|
|
|
|
|
await endpoint.end()
|
|
|
|
|
|
|
|
|
|
assert link.asked == [(protocol.AGENT_CANCEL, {"channel": 2}, node_client._CANCEL_TIMEOUT_S)]
|
|
|
|
|
assert endpoint.returncode is None, "the exit is the node's to report"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- where a node lives -------------------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
|
|
|
("kept", "want"),
|
|
|
|
|
[
|
|
|
|
|
(None, "~/.raven-node"),
|
|
|
|
|
("", "~/.raven-node"),
|
|
|
|
|
(" /srv/raven-node/ ", "/srv/raven-node"),
|
|
|
|
|
("~/elsewhere", "~/elsewhere"),
|
|
|
|
|
],
|
|
|
|
|
)
|
|
|
|
|
def test_the_node_dir_is_the_row_s_or_the_default(kept, want):
|
|
|
|
|
extra = {} if kept is None else {"node_dir": kept}
|
|
|
|
|
target = remote.Machine(id="box", display_name="Lab box", _row=row(**extra))
|
|
|
|
|
|
|
|
|
|
assert node_client.node_dir(target) == want
|
|
|
|
|
assert node_client.node_python(target, "0.1-abc") == f"{want}/0.1-abc/bin/python"
|
|
|
|
|
assert node_client.install_hint(target) == "raven ops connection install-node box"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- files both ways, through the node -------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_file_is_fetched_from_the_machine_whole_and_checked(box, pool, tmp_path):
|
|
|
|
|
box.node()
|
|
|
|
|
session = box.home / "raven-work" / "t1"
|
|
|
|
|
session.mkdir(parents=True)
|
|
|
|
|
roots = [f"{box.home}/raven-work"]
|
|
|
|
|
payload = os.urandom(protocol.CHUNK * 2 + 123)
|
|
|
|
|
(session / "report.bin").write_bytes(payload)
|
|
|
|
|
|
|
|
|
|
info = await pool.stat("box", f"{session}/report.bin", roots=roots, hash=True)
|
|
|
|
|
back = tmp_path / "back" / "report.bin"
|
|
|
|
|
got = await pool.pull_file("box", f"{session}/report.bin", back, roots=roots)
|
|
|
|
|
|
|
|
|
|
assert info["kind"] == "file" and info["size"] == len(payload)
|
|
|
|
|
assert info["sha256"] == hashlib.sha256(payload).hexdigest()
|
|
|
|
|
assert got == len(payload) and back.read_bytes() == payload
|
|
|
|
|
assert not back.with_name("report.bin.raven-part").exists()
|
|
|
|
|
with pytest.raises(NodeError, match="outside allowed directories"):
|
|
|
|
|
await pool.pull_file("box", f"{box.home}/elsewhere.bin", back, roots=roots)
|
|
|
|
|
assert len(box.launches()) == 1, "all of it on the one link"
|
|
|
|
|
|
|
|
|
|
|
feat(*): read, write and deliver files on a registered machine by address (#922)
## Summary
Third of four changes that let a Raven dispatch work to the machines in
its machine registry. #920 is merged; this branch is opened against main
ahead of #921, so the diff here includes #921's commits as well: review
from "read, write and deliver files on a registered machine by address"
on (the commits before it are #921's, carried here until it merges).
What this does:
- The file tools take a path on a registered machine as it is written
for scp -- `user@host:/path`, or `scp://user@host:port//path` when the
port is not 22 -- and answer it through the machine's daemon
(`raven/agent/tools/remote_files.py`, hooked from `filesystem.py` and
`file_search.py`): `read_file`, `list_dir`, `find` and `grep` run
Raven's own tools there and bring back the answer, so a read of fifty
lines of a large file moves those fifty lines; `write_file` and
`edit_file` write there, with the digest of that agent's own last read
-- the conversation's, or its sub-agent run's -- as the expected
state, so a file changed since it read it is refused and nothing is
written, and another agent's read does not stand in for its own. The
path is resolved against the registry
only, never this computer's ssh config, so one path names one machine
on any computer; one that matches no row is refused, not guessed.
- What is shown back as an address is only what the tool says about
paths -- its errors, its headers and footers, a write's account, a
diff's two header lines; a read's numbered lines, a search's matches
and context, and a diff's body come back as they are, so the model
edits against the bytes the file holds. The shell's refusal of typed
`ssh` to a registered machine (`raven/agent/tools/shell.py`,
`machine_exec.py`) reads the program in any spelling -- a path, a
backslash, and on a Windows host `ssh.exe`, `SSH.EXE` or a whole
`C:\...\ssh.exe`, with the backslashes the shell there keeps and
without the quotes it takes off (a quoted piece and a bare one next to
it making one word), so a quoted destination or port is matched like a
bare one -- and, with the lane on, names the file tools. `scp`, `sftp`
and `rsync` to a registered machine stay what they are on main: a copy
starts nothing there and passes exec's own question like any command,
and the file tools cannot carry a directory, a binary file or a large
one, so the copy stays the route for those. A drive-relative or
extended Windows path (`C:file`, `\\?\C:\...`, UNC) is this
computer's, not a machine nobody registered.
- Permissions (`raven/permissions/`): a path inside the directories
Raven's agents work in on that machine -- its session directory, its
`remoteCwd`, the row's listed `paths` -- is read without asking;
elsewhere a read asks once per machine and directory for the session.
A write there asks wherever it falls, as a write here does, and its
grant is for the one file, apart from any read. The daemon checks the
roots again on the machine, links included.
- `deliver_files` fetches one file by its address, whole and checked,
under a 20 MB cap, into `<workspace>/pulled/<machine>/<key>/<name>`,
the key a digest of the file's path on the machine, so two paths never
share a landing: not two files of one name from two directories, and
not two names this computer's filesystem reads as one (`Report.txt`
and `report.txt`, a trailing dot). The name beside the key is written
as one this computer keeps (a backslash, what Windows reserves in a
name, control characters and the percent sign itself as
percent-codes), and a landing that would not resolve under the
machine's delivery directory is refused before anything is made; a
bigger file is named with its size and read there instead. Only a file
under the directories this Raven's agents work in
on that machine is fetched: the tool asks nothing, and a read some
conversation was allowed is no evidence for this one, so a file
elsewhere is refused and read there instead. Nothing reclaims
`pulled/`: a delivery is opened from its landing for as long as its
record lasts, and a bound by size or age is left out of this change on
purpose.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_remote_file_tools.py
tests/test_permissions_remote_read.py tests/test_deliver_files_tool.py
tests/test_shell_machine_channel.py tests/test_ops_remote_path.py
tests/test_permissions_gate.py tests/test_import_cycle_budget.py
tests/test_node_server.py tests/test_node_client.py
tests/test_subagent_acp_remote.py tests/test_acp_remote.py
tests/test_cli_ops_connection_commands.py`
-- all passed on this tree (703 in the run that covered the first
eight groups, the node's own suite and the real-node cases among them).
Covered: each tool on a remote path answers as its local tool would,
through a stood-in link and a real daemon; a path outside the roots is
refused on both sides; a write with a stale digest is refused and
writes nothing; the address form on what the tool says about paths,
and a read's lines, a grep's matches and a diff's body untouched; the
shell's refusal of typed ssh to a registered machine, by host, by
alias, in a pipeline, as `ssh.exe` and `SSH.EXE`, and with a whole
Windows path or a quoted destination or port when the host is Windows,
a copy with scp, sftp or rsync not refused, and the file tools named
only with the lane on; a fetched name holding the
other platform's separators lands as one name under the machine's
directory and nothing lands elsewhere, and a landing that would
resolve outside fetches nothing; a Windows path form read here with the
lane on; the read
permission inside the roots and the question elsewhere, remembered per
machine and directory, a write asked wherever it falls and its grant
kept per file and apart from a read; a read's digest kept per agent,
so another agent's read lets no stale overwrite through and an agent
that never read the file expects nothing; a delivery fetched whole
under a key of its path (two files of one name, two names a
filesystem reads as one, and the percent-code pair all land apart),
refused over the cap and refused outside the roots whatever was read
before.
- `uv run pytest` on this tree after the merge with main -- 31210
passed,
231 skipped, 2 xfailed, 4 failed, every one of the four in a test that
needs what this machine lacks (`test_rpc_files.py`, the two simulation
cases and the kimi-code case that fail the same way on main here); the
groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Diff coverage of this change against main after the merge: 95.27% (805
of
845 executable changed lines), against the 90% gate. The node suite,
the remote-path tests and the real-node cases among the groups above run
on the merged tree.
- Real machines (two registered Linux machines): fifty lines from the
middle of a 5 MB file read through the daemon in 30 to 35 ms, byte for
byte what this computer reads; list, find and grep answered; a write
and an edit landed as the local tools would, and a write after the file
changed was refused; a path outside the roots refused; a file written
by an agent there fetched back by address. The shell's refusals are
covered by the tests above.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint`,
`ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
User docs follow once the lane is no longer debug-only.
## Risk
Behind the debug lane, as the two changes before it. With it on, a
remote
path reaches the machine only through its daemon, inside the roots each
call names and under the read and write questions above; the shell
cannot
run a command on a registered machine by address, and a copy to or from
it
with scp, sftp or rsync is the plain shell's behind exec's own question,
as
on main. A write refused for a stale
digest writes nothing. A delivery fetches at most 20 MB per file into
the
workspace, and only from the directories the agents work in on the
machine. Rollback is a plain revert.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#920 (merged); depends on #921.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 16:10:18 +08:00
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_a_cli_agent_s_input_travels_on_the_link_and_its_stdin_ends_when_told(box, pool):
|
|
|
|
|
box.node()
|
|
|
|
|
|
|
|
|
|
endpoint = await pool.open_channel("box", name="cat@box", command="cat", cwd="~", env={})
|
|
|
|
|
await endpoint.send_input("first line\nsecond line\n", eof=True)
|
|
|
|
|
|
|
|
|
|
lines = []
|
|
|
|
|
while raw := await endpoint.readline():
|
|
|
|
|
lines.append(raw.decode("utf-8").rstrip("\n"))
|
|
|
|
|
assert lines == ["first line", "second line"], "what went in came back, line by line"
|
|
|
|
|
assert await endpoint.wait() == 0, "its stdin closed, cat left on its own"
|
|
|
|
|
assert len(box.launches()) == 1, "on the node's ssh, like a frame"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_input_written_while_the_link_is_down_goes_out_in_order_once_it_is_back():
|
|
|
|
|
endpoint = ChannelEndpoint(_Client(), 1, target=_box(), agent="cli@box") # type: ignore[arg-type]
|
|
|
|
|
endpoint.detach("the link went")
|
|
|
|
|
await endpoint.write(b'{"a": 1}\n')
|
|
|
|
|
await endpoint.send_input("the prompt", eof=True)
|
|
|
|
|
back = _Client()
|
|
|
|
|
|
|
|
|
|
await endpoint.reattach(back, 7) # type: ignore[arg-type]
|
|
|
|
|
|
|
|
|
|
assert back.said == [
|
|
|
|
|
(protocol.AGENT_FRAME, {"channel": 7, "line": '{"a": 1}'}),
|
|
|
|
|
(protocol.AGENT_STDIN, {"channel": 7, "data": "the prompt", "eof": True}),
|
|
|
|
|
], "on the new channel, in the order they were written"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_input_for_an_agent_that_has_ended_or_whose_link_just_broke_is_refused_or_held():
|
|
|
|
|
endpoint = ChannelEndpoint(_Client(), 1, target=_box(), agent="cli@box") # type: ignore[arg-type]
|
|
|
|
|
endpoint.exited(0)
|
|
|
|
|
with pytest.raises(BrokenPipeError, match="has ended"):
|
|
|
|
|
await endpoint.send_input("late", eof=True)
|
|
|
|
|
|
|
|
|
|
class _Gone(_Client):
|
|
|
|
|
async def notify(self, method: str, params: dict[str, Any]) -> None:
|
|
|
|
|
raise AcpConnectionError("the ssh ended")
|
|
|
|
|
|
|
|
|
|
held = ChannelEndpoint(_Gone(), 1, target=_box(), agent="cli@box") # type: ignore[arg-type]
|
|
|
|
|
await held.send_input("the prompt", eof=True)
|
|
|
|
|
assert held._pending == [(protocol.AGENT_STDIN, {"data": "the prompt", "eof": True})], "kept for the next link"
|
|
|
|
|
|
|
|
|
|
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
async def test_frames_held_for_a_link_that_is_down_are_capped(tmp_path):
|
|
|
|
|
endpoint = ChannelEndpoint(_Client(), 1, target=_box(), agent="echo@box") # type: ignore[arg-type]
|
|
|
|
|
endpoint.detach("the link went")
|
|
|
|
|
for n in range(node_client._PENDING_FRAMES):
|
|
|
|
|
await endpoint.write(f"{n}\n".encode())
|
|
|
|
|
with pytest.raises(BrokenPipeError, match="down too long: 1000 frames wait"):
|
|
|
|
|
await endpoint.write(b"one more\n")
|
|
|
|
|
assert endpoint.detached == "the link went" and endpoint.returncode is None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_task_the_machine_no_longer_knows_ends_its_endpoint_when_the_link_is_back(tmp_path):
|
|
|
|
|
pool = NodePool()
|
|
|
|
|
gone = AcpRemoteError(protocol.AGENT_ATTACH, protocol.UNKNOWN_TASK, "no task 'echo@box-1' is running or recorded")
|
|
|
|
|
client = _live(pool, tmp_path, raises=gone)
|
|
|
|
|
endpoint = ChannelEndpoint(client, 1, target=_box(), agent="echo@box", task="echo@box-1") # type: ignore[arg-type]
|
|
|
|
|
endpoint.detach("the link went")
|
|
|
|
|
pool._parked["box"] = [endpoint]
|
|
|
|
|
|
|
|
|
|
await pool._reattach("box")
|
|
|
|
|
|
|
|
|
|
assert endpoint.returncode == node_client.LOST_RC
|
|
|
|
|
assert endpoint.ended() == (
|
|
|
|
|
"machine 'Lab box' (box) no longer runs 'echo@box' (task echo@box-1): no task 'echo@box-1' is running or recorded"
|
|
|
|
|
)
|
|
|
|
|
assert pool._parked.get("box", []) == [] and pool._nodes["box"].channels == {}
|
fix(node): an attach replays the agent's stderr past the line it last read (#977)
## Summary
Follow-up to LivXue's review note on #957 (the pick-up's binding at
`cli_agent.py:1393`). The machine's daemon kept an agent's stderr in the
task's record (`stderr.log`) but sent it only to the connection holding
the
task when a line was written, and `agent/attach` replayed `frames.jsonl`
alone. So what an agent said only on stderr while no connection held it
never
reached the host:
- **The session id a CLI names only on stderr** (hermes does). The
pick-up
#957 added binds a create's handle to the id in its output; for such a
CLI
it found none, warned "not resumable", and the next turn to the instance
opened a new session.
- **The reason a command failed.** A run picked up after this computer
stopped was reported as `exited 3` where the dispatch would have said
`exited 3: token expired`. This part predates #957.
- **The stderr written while a link was down**, on a run still being
watched, the same way.
The fix is in the node and the host's link to it (`raven/node/`):
- The daemon numbers stderr lines as it numbers frames (`agent/stderr`
carries `n`), and `agent/attach` takes an optional `stderr_after`: the
stderr lines past that one are replayed after the frames, under the lock
that forwards a new line, so none is sent twice or missed at the
handover.
It works the same for a live task, a finished one, and a record a daemon
that never ran the task reads. An attach that does not ask hears no
stderr,
as before; a `stderr_after` that is not a whole number is refused like a
bad `after`.
- The host's `ChannelEndpoint` counts the stderr lines it got
(`stderr_received`). A cli pick-up (`NodePool.attach`, a process that
read
none of them) asks from the first line; an acp pick-up from the line the
run's record kept as the prompt went out (`RemoteRun.stderr_received`,
follow-up commit below); a re-attach after a lost link
(`NodePool._reattach`) asks from the last one it read.
The cli dispatch and pick-up read stderr off the endpoint as before and
now
get all of it, so the reason and the session id are there. An acp
pick-up's
connection drains the same endpoint into the stderr tail it quotes in
its
errors and into its journal, so what the agent wrote during the turn
while
this computer was away reaches them too; the acp run record keeps the
stderr
line count as the prompt goes out, so the earlier prompts' lines are not
read again (follow-up commit below). No protocol bump:
the new fields are optional, and a node runs exactly the host's code
anyway
(protocol and digest are checked at every connection).
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `tests/test_node_stderr_replay.py`, new, 22 tests: the daemon numbers
and
keeps stderr lines; an attach replays exactly the lines past
`stderr_after`
(from a live task, a finished one, and a record a daemon that never ran
it
reads), none when not asked, none from a record with no `stderr.log`,
and
refuses a bad `stderr_after`; the endpoint counts by the node's numbers;
a
re-attach and a pick-up ask from the right line. End to end on the
stand-in machine (real daemon, stand-in ssh): a left cli run's reason on
stderr reaches its pick-up (said before and after this computer
stopped),
a left create that names its session only on stderr is bound to it by
the
pick-up (both orders), and a reason written while the link was down
reaches the run watching it. 18 fail on main (2c2718122); the 4 that
pass
there pin what does not change (no `stderr_after`, past the last line,
past the end, no `stderr.log`).
- Changed assertions: the four attach-parameter assertions in
`tests/test_node_client.py` and the forwarded-stderr one in
`tests/test_node_server.py` include the new fields.
- Full suite: 32065 passed, 173 skipped, 2 xfailed, 3 failed -- the same
3
that fail on main (2c2718122) on this computer when run alone there: a
host
without LibreOffice, and two simulation cases.
- Diff coverage: 100.00% (30/30 executable changed lines; `coverage
json`, then
`scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`).
- `ruff check`, `ruff format --check`, `ty check`, `lint-imports`,
pre-commit on the diff; `scripts/check_commit_messages.py`,
`scripts/check_source_language.py` and `scripts/check_large_files.py` on
`origin/main..HEAD`: clean.
- Docs: CONTEXT.md's Raven node entry, the protocol module's notes, and
the
machines guide's "Disconnects and restarts" section in English and
Chinese
now say the daemon keeps and replays stderr; the changelog has an entry
under Fixed. `tests/test_docs_site.py` and `tests/test_living_docs.py`
pass
(18), `mkdocs build --strict` passes.
- Real machine, ravenx2, an isolated node directory per tree. The host
process that dispatched was killed with SIGKILL and the run picked up by
a
new process; the link was dropped by killing the local ssh. The command
writes `token expired` (or `session slow-A1`) on stderr:
| Probe | main (2c2718122) | this branch |
|---|---|---|
| reason written after the host process was killed | `exited 3` |
`exited 3: token expired` |
| reason written before it was killed | `exited 3` | `exited 3: token
expired` |
| reason written while the link was down | `exited 3` | `exited 3: token
expired` |
| create left before its commit, session id only on stderr | "not
resumable", nothing bound | bound to `slow-A1`, no warning |
The machine was cleaned after: the probe's two daemons ended by pid, its
directory removed.
- The runs above are on main 2c2718122 (#957 merged). The branch was
then
rebased onto 400b30dfa, whose two new commits touch only TUI copy and
i18n; the node, docs and range checks were run again there (238 passed,
1
skipped). It merges cleanly with the open #964 and #965 (`git
merge-tree`).
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
- Follow-up commit e0468fa88, after the agent review's F1 (an acp
pick-up asked
for the stderr from the agent's first line, across every earlier prompt
of
its process): `RemoteRun.stderr_received`, taken in
`Tracker.prompt_sent`
off the endpoint's count, and passed through `NodePool.attach(...,
stderr_after=...)` by the pick-up; a cli pick-up keeps asking from 0; a
record from before the field reads with 0. Regression on the stand-in
machine
(`test_a_pick_up_reads_the_turn_s_own_stderr_and_not_what_the_prompts_before_it_wrote`):
two turns of 150 stderr lines, a third interrupted after its prompt, the
next start's pick-up -- before the change its tail held 200 lines, 50 of
them the second turn's; after it, the third turn's 150 and nothing else.
Unit tests pin the record, the pick-up's attach arguments and the
default
of 0, and the tracker's wiring. The related files 259 passed; full suite
32067 passed with the three failures this Mac always has, plus one test
whose stand-in `Tracker` lacked the new keyword, fixed in the same
commit.
Diff coverage of the changed lines: see CI.
## Risk
The replayed lines are the agent's own stderr, already kept in the
task's
owner-only record and already sent live to whichever connection held the
task; nothing new leaves the machine. `stderr_after` and `n` are
optional: an
attach without `stderr_after` behaves as before, and a line without a
number
counts on from the last. A cli pick-up replays the whole `stderr.log`,
as it
already replays every frame of the command; an acp pick-up replays the
lines
past the count its record kept as the prompt went out, so the replay is
bounded to the turn, and the connection keeps a bounded tail of it.
Rollback
is reverting the commit.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follow-up to #957, for this review note:
https://github.com/EverMind-AI/Raven/pull/957#discussion_r4282448689
---------
Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-17 21:08:32 +08:00
|
|
|
assert client.asked[-1][1] == {"channel": 1, "task": "echo@box-1", "after": 0, "stderr_after": 0}
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
await pool.close_all()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_an_attach_the_link_cannot_carry_parks_every_waiting_endpoint_again(tmp_path):
|
|
|
|
|
pool = NodePool(backoff_s=())
|
|
|
|
|
client = _live(pool, tmp_path, raises=AcpTimeoutError("no answer", method=protocol.AGENT_ATTACH))
|
|
|
|
|
first = ChannelEndpoint(client, 1, target=_box(), agent="a@box") # type: ignore[arg-type]
|
|
|
|
|
second = ChannelEndpoint(client, 2, target=_box(), agent="b@box") # type: ignore[arg-type]
|
|
|
|
|
for endpoint in (first, second):
|
|
|
|
|
endpoint.detach("the link went")
|
|
|
|
|
pool._parked["box"] = [first, second]
|
|
|
|
|
|
|
|
|
|
await pool._reattach("box")
|
|
|
|
|
|
|
|
|
|
assert first.returncode is None and second.returncode is None
|
|
|
|
|
assert pool._parked["box"] == [first, second], "the one tried and the one not yet reached both wait"
|
|
|
|
|
assert not client.alive and "box" not in pool._nodes
|
|
|
|
|
assert len(client.asked) == 1, "the second was not tried on a link that had just failed"
|
|
|
|
|
assert pool.status("box").state == "lost" and pool.status("box").agents == 2
|
|
|
|
|
await pool.close_all()
|
|
|
|
|
|
|
|
|
|
|
feat(*): pick up a turn left on a registered machine when this computer stops (#934)
## Summary
Registered machines are now available without a debug environment
switch.
`RAVEN_DEBUG_MACHINES` previously defaulted to off: it kept
machine-backed
agents out of the dispatch table, refused node/installer access, hid
remote
preset destinations, and treated file paths locally while off. It was a
trial feature gate, not a verbose-logging option. At the owner's
request,
this PR removes the variable and all of those checks. Normal startup and
restart recovery use the configured machine entries; even a leftover
value
of `0` is ignored. Machine registration, SSH authentication, file
permission
prompts, and concurrency limits still apply.
Fourth change in the series that lets a Raven dispatch work to the
machines in its machine registry. #920, #921 (merged through #922) and
#922 are on main; this change stands on main alone now, and the diff is
its own.
What this does: a turn running on a registered machine survives this
computer stopping, and the next start picks it up and reports it.
- While a turn runs on a machine, this computer remembers what the
machine does not know (`raven/acp_client/remote_runs.py`): one
owner-only file per run under `<raven home>/machines/runs/`, with the
machine's registry id, the task id its daemon runs the agent under, the
agent, the session key, the instance handle, the dispatch id, the ACP
session, the id of the prompt whose answer is awaited, the ids this
side has used, how many of the agent's frames were read here, the
agent's answer to `initialize` and the connection's launch key. Kept
current after every frame the connection has dealt with, removed when
the turn ends here. No address is in it.
- When this process stops -- the gateway, `serve`, the TUI, the page
stack -- the
agents on machines stay with their daemons
(`remote.leave_agents_running()` before the drain): a cancelled turn
on a machine sends no `session/cancel`, closing its connection ends the
endpoint only, and the manager leaves the dispatch's record open rather
than finishing it as cancelled. A hard kill or `/restart` leaves the
same record, written as the turn ran.
- At the next start -- the gateway once its generation is bound; the
`serve`, TUI and page stacks once --
`raven/agent/subagent/resume_remote.py`
picks every run up: `task/list` on the machine, `agent/attach` from the
frame last read (`NodePool.attach`), the connection adopted into the
pool with request ids carried on past the other process's
(`AcpConnectionPool.adopt`, `AcpClient.continue_ids_from`), the
prompt's answer awaited (`AcpClient.await_response`), the reply built
as a turn ended here is -- the partial notice when the agent stopped
early, the machine note saying it was picked up after this computer
restarted -- then the dispatch's record finished, the registry row and
the node claim closed, and the result announced into the conversation
that asked, as the manager announces a turn that ended here
(`raven/acp_client/remote_resume.py`). A machine that cannot be asked
keeps its record for a later start; a run the machine no longer has,
one whose agent ended while this computer was away, one whose agent
left the agent table, and one whose turn says nothing are announced as
errors that say so. The adopted agent process serves later dispatches
of its entry, as one started here would.
- Seams: `AcpClient.request(sent=)` hears the id of what goes out;
`ChannelEndpoint.watchers` hear the frames the connection has dealt
with (`consumed`, called by the read loop); `ChannelEndpoint.stays`
reads the leaving flag.
- A cli entry's run on a machine is recorded while it runs (kind `cli`,
no ACP part) and left the same way, the manager reading the dispatch
as left; the entry picks it up (`CliAgentBackend.pick_up`): the daemon
replays the command's whole output from its first line, the exit code
ends it, and the reply closes with the machine note.
- The record carries the task as the model wrote it, and -- once the
manager has read the run as left -- the dispatch's origin and summary
(`remote_runs.annotate`): the announce after a restart names the
instance, node and tool call as the announce here does and quotes the
task as authored, never the rendered prompt on disk; a host killed
outright gets an origin made up from the spawn record and the run.
- The record goes last, once the outcome has reached its conversation:
a conclusion that fails, a stop under it, or an announce the scheduler
refuses (`_announce_result` says so now) leaves the record, its claim
released, for the next start to try again.
- A run that returned here keeps its record the same way, until the
manager's announce of the reply was taken: the manager dispatches with
`reported_by_caller=True`, the backend then leaves the record behind
with the reply instead of removing it as it returns, and the manager
removes it after the announce (`remote_runs.forget_dispatch`) -- or,
when the draining scheduler refused the announce, annotates it with
the dispatch's origin and summary for the next start to report. A host
killed between the agent's answer and the announce therefore still has
the way back to the run. A caller that takes the reply as its own
result (a graph node, a probe, a direct chat) dispatches without the
keyword, and the record goes as it did. The record's frame count stops
at the prompt: the turn's own frames, the chunks before the answer as
much as the answer, are never checkpointed, so a pick-up replays the
turn whole and the reply it reports is complete; a record that had
moved past the answer would have resumed after it and waited forever.
Not in this change: a run from the DAG lane has no spawn record to
finish; it is announced from the origin the manager added to its record
when it was left, or, for a host killed outright, from the run's record
alone. The Tasks page reads the finished record on its next load rather
than live.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Machine availability and restart checks after removing the trial switch:
- New default-behavior regressions were run before the removal: 8
failed,
with the enabled-value control passing. After removal, all 9 passed,
including ACP/CLI recovery through the real AgentRegistry with the
variable absent or set to `0`.
- `uv run --no-sync pytest tests/test_acp_remote.py
tests/test_node_client.py tests/test_node_server.py
tests/test_node_install.py tests/test_ops_remote_path.py
tests/test_permissions_remote_read.py tests/test_remote_file_tools.py
tests/test_deliver_files_tool.py tests/test_raven_config_tool.py
tests/test_rpc_subagents.py tests/test_shell_machine_channel.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_resume_remote.py tests/test_remote_resume.py
tests/test_remote_runs.py tests/test_subagent_manager.py
tests/test_cli_tui_commands.py tests/test_rpc_schema_match.py
tests/test_config_schema.py -n 4 --idle-ceiling-strict --cov=raven
--cov-branch --cov-report=json:<coverage-json> --cov-report= --tb=short`
-- 1799 passed on the combined tree, including the latest published
result-retention fix. An earlier run had a missing stdio response in the
minimal-node dependency test and an idle-gate hit; both passed a focused
recheck, and this final complete selection passed with the strict gate.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 92.92% (656/706 changed executable lines) for this PR; the
gate-removal
commit alone covers 2/2 new executable lines (the remainder deletes
gates
and updates descriptions).
- `UV_NO_SYNC=1 make lint-python lint-imports lint-deps lint-types`, the
commit/source-language/large-file scope gates and `git diff --check`
pass.
Both RPC generators' `--check` runs pass; `npm test --
scripts/gates/fixture-shape.test.mjs`
from `ui-web` passes all five tests.
- Real Linux machine, isolated node installations, separate dispatch and
restart processes: before removal, a task started with the trial switch
on was reported as an absent agent and its recovery record deleted when
restarted without it. After removal, both absent and old `0` settings
recover the full CLI result and read its output file by address. The
absent-variable case kills the relay before host shutdown. The combined
final tree was reinstalled and both scenarios repeated successfully.
- With the variable absent, 12 further timing/boundary probes pass:
early
CLI exit and blank lines, relay loss before heartbeat with normal-call
recovery, tunnel loss, ACP/CLI/OpenAI cancellation during disconnection,
11 launches admitting 8 and refusing 3, occupied tunnel port, and an
unwritable prompt file. Three attachment, POSIX-command and IPv6 checks
also pass; Windows command parsing is simulated, not native Windows.
No test task or tunnel remains; the dedicated node directories are
removed.
- CONTEXT.md, configuration and RPC descriptions, transport/path
docstrings,
both collaboration guides and the owner's design notes were checked
against the code. The guides explain the former trial switch and normal
startup without it. The existing SSH, permission and limit tests
continue
to exercise their checks with no test-only switch setup.
- `env -u RAVEN_DEBUG_MACHINES uv run --no-sync pytest -n 4 --tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
-- 31345 passed, 230 skipped, 2 xfailed, 5 failed. A separate
current-main
checkout reproduces four failures: CLI color rendering on this host,
LibreOffice detection, and the two simulation cases. The remaining
failure
was the direct-chat test stub rejecting `reported_by_caller` from the
latest result-retention change; the published wrapper fix now forwards
only keywords declared by its fake backend. The full
suite was not rerun for that test-only correction. Final direct-chat and
manager recheck: 277 passed with strict idle accounting. Full-PR
changed-line coverage from this
run is 96.18% (679/706).
Earlier verification:
- `uv run pytest tests/test_remote_runs.py tests/test_remote_resume.py
tests/test_resume_remote.py tests/test_acp_client_resume.py
tests/test_node_client.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_cli_tui_commands.py
tests/test_subagent_manager.py tests/test_rpc_bootstrap.py
tests/test_cli_gateway_commands.py tests/test_acp_client_launch.py
tests/test_subagent_acp.py tests/test_scope_canon.py
tests/test_l4_entrances.py`
-- all passed (605 passed). Covered: the record written owner-only, kept
current
per frame and per prompt, removed when the turn ends here and kept when
this process leaves; a dispatch left at shutdown neither finished nor
announced by the manager; an endpoint ended while leaving leaves the
agent with the daemon (real daemon); a task picked up from the frame
last read, with the watchers hearing each frame; a pick-up the machine
refuses or cannot carry; an answer awaited for a request another
process sent, ids carried on, no `session/cancel` while leaving; the
reply read off replayed frames with the other sessions' frames
ignored, a turn cut short marked partial, an unreachable machine
keeping the record, a run the machine no longer has, a prompt never
sent, an agent that ended while this computer was away, an answer
waited out, a turn that said nothing, a pick-up beside a live
connection closing its own; the manager-side conclusion: record
finished and announced, an error announced and the record failed, an
unreachable machine keeping the record, an agent off the table, a
failure inside one pick-up not stopping the next, a run with no
conversation, the once-per-process scheduling; a cli run recorded
while it runs and left at shutdown, picked up whole on the real daemon
behind the stand-in ssh, its pick-up's errors said (a non-zero exit, a
machine that no longer has it, a link down, this computer leaving), a
cancel that is not a leave ending the command with no record left; the
TUI's stop saying the agents stay before the cancel and its start
scheduling the pick-up; the record annotated with the dispatch's
origin and summary when the manager reads it as left, the announce
quoting the task as authored and naming instance, node and tool call,
with and without the manager's fields; the record outliving a refused
announce and a failed conclusion, claim released; a cli record read
without an ACP session, an older record read with defaults; the record
moving with the frames before the prompt and not past it, a dispatch's
record forgotten by the manager's id with its claim, the manager
dispatching with `reported_by_caller` and forgetting the record after
an announce that was taken and annotating it after one that was
refused; on the real daemon behind the stand-in ssh, an acp reply and
a cli output returned with the record kept and the host "stopped"
before the announce, the next start reading the same reply again
whole, and a dispatch without the keyword leaving no record. End to end
on the real
daemon behind the stand-in ssh: a turn dispatched, this process
"stopped" the way the gateway stops, the agent still running on the
daemon, then a fresh pick-up reading its answer, finishing the record
and announcing it, with the agent's process the pool's connection
again.
- `uv run pytest` on this tree, at an earlier revision of this change --
29270 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine; diff coverage of this
change against its base: 97.45% (153 of 157 executable changed lines),
and 94.67% for the four changes against main; `lint-imports` 10 kept,
0 broken; `ty` clean.
- Real machine (a registered Linux machine with Claude Code signed in,
node reinstalled with this tree): process A dispatched a 25 s task,
saw the prompt out after 2.4 s and the record keep up with the frames
(6, then 10), then stopped the way the gateway stops 5.4 s after the
dispatch, leaving the record and the agent running there; process B,
started fresh, found the record and the running task, picked it up
in 2.4 s, announced the agent's reply to the conversation with the
machine note, finished the record, removed the run's record, held the
agent's process as the pool's connection, and closing the pool ended
the agent on the machine. This revision, the cli lane with the code
before and after, the node reinstalled for each: a cli entry
`sh -c 'echo $$ > pid.txt; sleep 20; echo done; cat'` was dispatched
and the host stopped the gateway's way 2.7 s later. Before: no
record, the command left running on the machine, and the next start
found nothing to pick up. After: the record kept with the task as
authored and the manager's origin, the command running on, and the
next start picked it up in 16.8 s (the sleep), announcing `done` and
`hi` with the instance, node and tool call and the machine note. The
same with the link killed before the stop: recorded, left, picked up.
With the machine's daemon ended between the stop and the pick-up, it
ended its groups on its way out, the task's record there said exit
-9, and the pick-up announced that as the run's error in 0.1 s,
leaving nothing on the machine.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint`,
`ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese agent-collaboration guides explain normal
startup,
registered-machine setup and recovery, and the removed trial switch.
## Risk
Configured machine entries are available by default; entries without a
`machine` remain local. A remote-looking file path is resolved against
the
registry even without an environment variable; use an explicit local
path
such as `./a:b` for a local filename containing a colon. A formerly
hidden
machine entry is now available for dispatch, and an old variable set to
`0`
no longer disables it. Stopping Raven no longer cancels a turn
running on a machine; it goes on there until the agent finishes or is
cancelled, and its answer arrives in the conversation after the next
start, as a sub-agent result that triggers a turn. One record per run in
the raven home, owner-only, removed when the run is concluded; a machine
that cannot be reached keeps it, so a run is reported at most once and
never lost quietly. The adopted connection answers requests with ids
past the ones the stopped process used, so a replayed frame resolves
nothing sent after the restart. Rollback is a plain revert; a record
left
behind is then an inert file.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#920 and #922 (merged); #921 was carried by #922.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-14 21:34:19 +08:00
|
|
|
async def test_a_pick_up_the_machine_refuses_or_cannot_carry_is_one_sentence_each(tmp_path):
|
|
|
|
|
pool = NodePool(backoff_s=())
|
|
|
|
|
gone = AcpRemoteError(protocol.AGENT_ATTACH, protocol.UNKNOWN_TASK, "no task 't' is running or recorded")
|
|
|
|
|
_live(pool, tmp_path, "box", raises=gone)
|
|
|
|
|
with pytest.raises(NodeError, match=r"machine 'Lab box' \(box\) no longer runs 'echo@box' \(task t\): no task"):
|
|
|
|
|
await pool.attach("box", "t", name="echo@box")
|
|
|
|
|
assert pool._nodes["box"].channels == {}, "the channel opened for it is closed again"
|
|
|
|
|
|
|
|
|
|
slow = _live(pool, tmp_path, "slow", raises=AcpTimeoutError("no answer", method=protocol.AGENT_ATTACH))
|
|
|
|
|
with pytest.raises(NodeError, match=r"did not answer agent/attach"):
|
|
|
|
|
await pool.attach("slow", "t", name="echo@box")
|
|
|
|
|
assert not slow.alive, "a link that does not answer is lost"
|
|
|
|
|
|
|
|
|
|
dead = _live(pool, tmp_path, "dead", raises=AcpConnectionError("connection ended (exit 255)"))
|
|
|
|
|
with pytest.raises(NodeError, match=r"machine 'Lab box' \(dead\)"):
|
|
|
|
|
await pool.attach("dead", "t", name="echo@box")
|
|
|
|
|
assert not dead.alive
|
|
|
|
|
await pool.close_all()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_watcher_that_fails_is_named_and_the_frames_go_on(tmp_path):
|
|
|
|
|
from loguru import logger
|
|
|
|
|
|
|
|
|
|
endpoint = ChannelEndpoint(_Client(), 1, target=_box(), agent="echo@box") # type: ignore[arg-type]
|
|
|
|
|
heard: list[int] = []
|
|
|
|
|
|
|
|
|
|
def bad(n: int) -> None:
|
|
|
|
|
raise RuntimeError("no disk")
|
|
|
|
|
|
|
|
|
|
endpoint.watchers.extend([bad, heard.append])
|
|
|
|
|
records: list[str] = []
|
|
|
|
|
sink = logger.add(lambda message: records.append(message.record["message"]), level="WARNING")
|
|
|
|
|
try:
|
|
|
|
|
endpoint.feed_frame('{"jsonrpc":"2.0","id":1,"result":{}}', 1)
|
|
|
|
|
assert await endpoint.readline() == b'{"jsonrpc":"2.0","id":1,"result":{}}\n'
|
|
|
|
|
endpoint.consumed()
|
|
|
|
|
finally:
|
|
|
|
|
logger.remove(sink)
|
|
|
|
|
|
|
|
|
|
assert heard == [1] and any("a watcher of task" in message and "no disk" in message for message in records)
|
|
|
|
|
|
|
|
|
|
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
# --- a link replaced by an ordinary call brings its agents along ------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_call_that_replaces_a_dead_link_attaches_the_parked_agents_again(tmp_path, monkeypatch):
|
|
|
|
|
"""The transport died before the heartbeat saw it: the next call relaunches the link, and the agents come with it."""
|
|
|
|
|
pool = NodePool(heartbeat_s=60.0)
|
|
|
|
|
client = _live(pool, tmp_path)
|
|
|
|
|
endpoint = ChannelEndpoint(client, 1, target=_box(), agent="echo@box") # type: ignore[arg-type]
|
|
|
|
|
pool._nodes["box"].channels[1] = endpoint
|
|
|
|
|
again = _Client()
|
|
|
|
|
|
|
|
|
|
async def launch(machine_id: str) -> _Node:
|
|
|
|
|
return _Node(again, _box(), tmp_path / "box.log", {"home": "/h"}, {}) # type: ignore[arg-type]
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(pool, "_launch", launch)
|
|
|
|
|
client.alive = False
|
|
|
|
|
|
|
|
|
|
assert await pool.call("box", protocol.PING, {}) == {"text": "ok"}
|
|
|
|
|
|
|
|
|
|
assert pool._parked.get("box", []) == [] and endpoint.detached is None
|
fix(node): an attach replays the agent's stderr past the line it last read (#977)
## Summary
Follow-up to LivXue's review note on #957 (the pick-up's binding at
`cli_agent.py:1393`). The machine's daemon kept an agent's stderr in the
task's record (`stderr.log`) but sent it only to the connection holding
the
task when a line was written, and `agent/attach` replayed `frames.jsonl`
alone. So what an agent said only on stderr while no connection held it
never
reached the host:
- **The session id a CLI names only on stderr** (hermes does). The
pick-up
#957 added binds a create's handle to the id in its output; for such a
CLI
it found none, warned "not resumable", and the next turn to the instance
opened a new session.
- **The reason a command failed.** A run picked up after this computer
stopped was reported as `exited 3` where the dispatch would have said
`exited 3: token expired`. This part predates #957.
- **The stderr written while a link was down**, on a run still being
watched, the same way.
The fix is in the node and the host's link to it (`raven/node/`):
- The daemon numbers stderr lines as it numbers frames (`agent/stderr`
carries `n`), and `agent/attach` takes an optional `stderr_after`: the
stderr lines past that one are replayed after the frames, under the lock
that forwards a new line, so none is sent twice or missed at the
handover.
It works the same for a live task, a finished one, and a record a daemon
that never ran the task reads. An attach that does not ask hears no
stderr,
as before; a `stderr_after` that is not a whole number is refused like a
bad `after`.
- The host's `ChannelEndpoint` counts the stderr lines it got
(`stderr_received`). A cli pick-up (`NodePool.attach`, a process that
read
none of them) asks from the first line; an acp pick-up from the line the
run's record kept as the prompt went out (`RemoteRun.stderr_received`,
follow-up commit below); a re-attach after a lost link
(`NodePool._reattach`) asks from the last one it read.
The cli dispatch and pick-up read stderr off the endpoint as before and
now
get all of it, so the reason and the session id are there. An acp
pick-up's
connection drains the same endpoint into the stderr tail it quotes in
its
errors and into its journal, so what the agent wrote during the turn
while
this computer was away reaches them too; the acp run record keeps the
stderr
line count as the prompt goes out, so the earlier prompts' lines are not
read again (follow-up commit below). No protocol bump:
the new fields are optional, and a node runs exactly the host's code
anyway
(protocol and digest are checked at every connection).
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `tests/test_node_stderr_replay.py`, new, 22 tests: the daemon numbers
and
keeps stderr lines; an attach replays exactly the lines past
`stderr_after`
(from a live task, a finished one, and a record a daemon that never ran
it
reads), none when not asked, none from a record with no `stderr.log`,
and
refuses a bad `stderr_after`; the endpoint counts by the node's numbers;
a
re-attach and a pick-up ask from the right line. End to end on the
stand-in machine (real daemon, stand-in ssh): a left cli run's reason on
stderr reaches its pick-up (said before and after this computer
stopped),
a left create that names its session only on stderr is bound to it by
the
pick-up (both orders), and a reason written while the link was down
reaches the run watching it. 18 fail on main (2c2718122); the 4 that
pass
there pin what does not change (no `stderr_after`, past the last line,
past the end, no `stderr.log`).
- Changed assertions: the four attach-parameter assertions in
`tests/test_node_client.py` and the forwarded-stderr one in
`tests/test_node_server.py` include the new fields.
- Full suite: 32065 passed, 173 skipped, 2 xfailed, 3 failed -- the same
3
that fail on main (2c2718122) on this computer when run alone there: a
host
without LibreOffice, and two simulation cases.
- Diff coverage: 100.00% (30/30 executable changed lines; `coverage
json`, then
`scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`).
- `ruff check`, `ruff format --check`, `ty check`, `lint-imports`,
pre-commit on the diff; `scripts/check_commit_messages.py`,
`scripts/check_source_language.py` and `scripts/check_large_files.py` on
`origin/main..HEAD`: clean.
- Docs: CONTEXT.md's Raven node entry, the protocol module's notes, and
the
machines guide's "Disconnects and restarts" section in English and
Chinese
now say the daemon keeps and replays stderr; the changelog has an entry
under Fixed. `tests/test_docs_site.py` and `tests/test_living_docs.py`
pass
(18), `mkdocs build --strict` passes.
- Real machine, ravenx2, an isolated node directory per tree. The host
process that dispatched was killed with SIGKILL and the run picked up by
a
new process; the link was dropped by killing the local ssh. The command
writes `token expired` (or `session slow-A1`) on stderr:
| Probe | main (2c2718122) | this branch |
|---|---|---|
| reason written after the host process was killed | `exited 3` |
`exited 3: token expired` |
| reason written before it was killed | `exited 3` | `exited 3: token
expired` |
| reason written while the link was down | `exited 3` | `exited 3: token
expired` |
| create left before its commit, session id only on stderr | "not
resumable", nothing bound | bound to `slow-A1`, no warning |
The machine was cleaned after: the probe's two daemons ended by pid, its
directory removed.
- The runs above are on main 2c2718122 (#957 merged). The branch was
then
rebased onto 400b30dfa, whose two new commits touch only TUI copy and
i18n; the node, docs and range checks were run again there (238 passed,
1
skipped). It merges cleanly with the open #964 and #965 (`git
merge-tree`).
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
- Follow-up commit e0468fa88, after the agent review's F1 (an acp
pick-up asked
for the stderr from the agent's first line, across every earlier prompt
of
its process): `RemoteRun.stderr_received`, taken in
`Tracker.prompt_sent`
off the endpoint's count, and passed through `NodePool.attach(...,
stderr_after=...)` by the pick-up; a cli pick-up keeps asking from 0; a
record from before the field reads with 0. Regression on the stand-in
machine
(`test_a_pick_up_reads_the_turn_s_own_stderr_and_not_what_the_prompts_before_it_wrote`):
two turns of 150 stderr lines, a third interrupted after its prompt, the
next start's pick-up -- before the change its tail held 200 lines, 50 of
them the second turn's; after it, the third turn's 150 and nothing else.
Unit tests pin the record, the pick-up's attach arguments and the
default
of 0, and the tracker's wiring. The related files 259 passed; full suite
32067 passed with the three failures this Mac always has, plus one test
whose stand-in `Tracker` lacked the new keyword, fixed in the same
commit.
Diff coverage of the changed lines: see CI.
## Risk
The replayed lines are the agent's own stderr, already kept in the
task's
owner-only record and already sent live to whichever connection held the
task; nothing new leaves the machine. `stderr_after` and `n` are
optional: an
attach without `stderr_after` behaves as before, and a line without a
number
counts on from the last. A cli pick-up replays the whole `stderr.log`,
as it
already replays every frame of the command; an acp pick-up replays the
lines
past the count its record kept as the prompt went out, so the replay is
bounded to the turn, and the connection keeps a bounded tail of it.
Rollback
is reverting the commit.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follow-up to #957, for this review note:
https://github.com/EverMind-AI/Raven/pull/957#discussion_r4282448689
---------
Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-17 21:08:32 +08:00
|
|
|
assert (
|
|
|
|
|
protocol.AGENT_ATTACH,
|
|
|
|
|
{"channel": 1, "task": endpoint.task, "after": 0, "stderr_after": 0},
|
|
|
|
|
) in [a[:2] for a in again.asked]
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
assert pool._nodes["box"].client is again and pool._nodes["box"].channels[1] is endpoint
|
|
|
|
|
await pool.close_all()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_relaunch_that_fails_under_a_call_still_reconnects_for_the_parked_agents(tmp_path, monkeypatch):
|
|
|
|
|
"""The link died unnoticed and the call's own relaunch failed: the agents wait on the backoff, as after a heartbeat loss."""
|
|
|
|
|
pool = NodePool(heartbeat_s=60.0, backoff_s=(0.01, 0.01))
|
|
|
|
|
client = _live(pool, tmp_path)
|
|
|
|
|
endpoint = ChannelEndpoint(client, 1, target=_box(), agent="echo@box") # type: ignore[arg-type]
|
|
|
|
|
pool._nodes["box"].channels[1] = endpoint
|
|
|
|
|
tried: list[str] = []
|
|
|
|
|
again = _Client()
|
|
|
|
|
|
|
|
|
|
async def launch(machine_id: str) -> _Node:
|
|
|
|
|
tried.append(machine_id)
|
|
|
|
|
if len(tried) == 1:
|
|
|
|
|
raise NodeError("machine 'Lab box' (box) could not be reached")
|
|
|
|
|
return _Node(again, _box(), tmp_path / "box.log", {"home": "/h"}, {}) # type: ignore[arg-type]
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(pool, "_launch", launch)
|
|
|
|
|
client.alive = False
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError, match="could not be reached"):
|
|
|
|
|
await pool.call("box", protocol.PING, {})
|
|
|
|
|
assert pool.waiting("box") == 1 and pool.status("box").state == "lost"
|
|
|
|
|
assert "box" in pool._reconnects, (
|
|
|
|
|
"an agent waits: the link is tried again on the backoff, not only by the next call"
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
await _until(lambda: endpoint.detached is None and len(tried) == 2)
|
fix(node): an attach replays the agent's stderr past the line it last read (#977)
## Summary
Follow-up to LivXue's review note on #957 (the pick-up's binding at
`cli_agent.py:1393`). The machine's daemon kept an agent's stderr in the
task's record (`stderr.log`) but sent it only to the connection holding
the
task when a line was written, and `agent/attach` replayed `frames.jsonl`
alone. So what an agent said only on stderr while no connection held it
never
reached the host:
- **The session id a CLI names only on stderr** (hermes does). The
pick-up
#957 added binds a create's handle to the id in its output; for such a
CLI
it found none, warned "not resumable", and the next turn to the instance
opened a new session.
- **The reason a command failed.** A run picked up after this computer
stopped was reported as `exited 3` where the dispatch would have said
`exited 3: token expired`. This part predates #957.
- **The stderr written while a link was down**, on a run still being
watched, the same way.
The fix is in the node and the host's link to it (`raven/node/`):
- The daemon numbers stderr lines as it numbers frames (`agent/stderr`
carries `n`), and `agent/attach` takes an optional `stderr_after`: the
stderr lines past that one are replayed after the frames, under the lock
that forwards a new line, so none is sent twice or missed at the
handover.
It works the same for a live task, a finished one, and a record a daemon
that never ran the task reads. An attach that does not ask hears no
stderr,
as before; a `stderr_after` that is not a whole number is refused like a
bad `after`.
- The host's `ChannelEndpoint` counts the stderr lines it got
(`stderr_received`). A cli pick-up (`NodePool.attach`, a process that
read
none of them) asks from the first line; an acp pick-up from the line the
run's record kept as the prompt went out (`RemoteRun.stderr_received`,
follow-up commit below); a re-attach after a lost link
(`NodePool._reattach`) asks from the last one it read.
The cli dispatch and pick-up read stderr off the endpoint as before and
now
get all of it, so the reason and the session id are there. An acp
pick-up's
connection drains the same endpoint into the stderr tail it quotes in
its
errors and into its journal, so what the agent wrote during the turn
while
this computer was away reaches them too; the acp run record keeps the
stderr
line count as the prompt goes out, so the earlier prompts' lines are not
read again (follow-up commit below). No protocol bump:
the new fields are optional, and a node runs exactly the host's code
anyway
(protocol and digest are checked at every connection).
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `tests/test_node_stderr_replay.py`, new, 22 tests: the daemon numbers
and
keeps stderr lines; an attach replays exactly the lines past
`stderr_after`
(from a live task, a finished one, and a record a daemon that never ran
it
reads), none when not asked, none from a record with no `stderr.log`,
and
refuses a bad `stderr_after`; the endpoint counts by the node's numbers;
a
re-attach and a pick-up ask from the right line. End to end on the
stand-in machine (real daemon, stand-in ssh): a left cli run's reason on
stderr reaches its pick-up (said before and after this computer
stopped),
a left create that names its session only on stderr is bound to it by
the
pick-up (both orders), and a reason written while the link was down
reaches the run watching it. 18 fail on main (2c2718122); the 4 that
pass
there pin what does not change (no `stderr_after`, past the last line,
past the end, no `stderr.log`).
- Changed assertions: the four attach-parameter assertions in
`tests/test_node_client.py` and the forwarded-stderr one in
`tests/test_node_server.py` include the new fields.
- Full suite: 32065 passed, 173 skipped, 2 xfailed, 3 failed -- the same
3
that fail on main (2c2718122) on this computer when run alone there: a
host
without LibreOffice, and two simulation cases.
- Diff coverage: 100.00% (30/30 executable changed lines; `coverage
json`, then
`scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`).
- `ruff check`, `ruff format --check`, `ty check`, `lint-imports`,
pre-commit on the diff; `scripts/check_commit_messages.py`,
`scripts/check_source_language.py` and `scripts/check_large_files.py` on
`origin/main..HEAD`: clean.
- Docs: CONTEXT.md's Raven node entry, the protocol module's notes, and
the
machines guide's "Disconnects and restarts" section in English and
Chinese
now say the daemon keeps and replays stderr; the changelog has an entry
under Fixed. `tests/test_docs_site.py` and `tests/test_living_docs.py`
pass
(18), `mkdocs build --strict` passes.
- Real machine, ravenx2, an isolated node directory per tree. The host
process that dispatched was killed with SIGKILL and the run picked up by
a
new process; the link was dropped by killing the local ssh. The command
writes `token expired` (or `session slow-A1`) on stderr:
| Probe | main (2c2718122) | this branch |
|---|---|---|
| reason written after the host process was killed | `exited 3` |
`exited 3: token expired` |
| reason written before it was killed | `exited 3` | `exited 3: token
expired` |
| reason written while the link was down | `exited 3` | `exited 3: token
expired` |
| create left before its commit, session id only on stderr | "not
resumable", nothing bound | bound to `slow-A1`, no warning |
The machine was cleaned after: the probe's two daemons ended by pid, its
directory removed.
- The runs above are on main 2c2718122 (#957 merged). The branch was
then
rebased onto 400b30dfa, whose two new commits touch only TUI copy and
i18n; the node, docs and range checks were run again there (238 passed,
1
skipped). It merges cleanly with the open #964 and #965 (`git
merge-tree`).
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
- Follow-up commit e0468fa88, after the agent review's F1 (an acp
pick-up asked
for the stderr from the agent's first line, across every earlier prompt
of
its process): `RemoteRun.stderr_received`, taken in
`Tracker.prompt_sent`
off the endpoint's count, and passed through `NodePool.attach(...,
stderr_after=...)` by the pick-up; a cli pick-up keeps asking from 0; a
record from before the field reads with 0. Regression on the stand-in
machine
(`test_a_pick_up_reads_the_turn_s_own_stderr_and_not_what_the_prompts_before_it_wrote`):
two turns of 150 stderr lines, a third interrupted after its prompt, the
next start's pick-up -- before the change its tail held 200 lines, 50 of
them the second turn's; after it, the third turn's 150 and nothing else.
Unit tests pin the record, the pick-up's attach arguments and the
default
of 0, and the tracker's wiring. The related files 259 passed; full suite
32067 passed with the three failures this Mac always has, plus one test
whose stand-in `Tracker` lacked the new keyword, fixed in the same
commit.
Diff coverage of the changed lines: see CI.
## Risk
The replayed lines are the agent's own stderr, already kept in the
task's
owner-only record and already sent live to whichever connection held the
task; nothing new leaves the machine. `stderr_after` and `n` are
optional: an
attach without `stderr_after` behaves as before, and a line without a
number
counts on from the last. A cli pick-up replays the whole `stderr.log`,
as it
already replays every frame of the command; an acp pick-up replays the
lines
past the count its record kept as the prompt went out, so the replay is
bounded to the turn, and the connection keeps a bounded tail of it.
Rollback
is reverting the commit.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follow-up to #957, for this review note:
https://github.com/EverMind-AI/Raven/pull/957#discussion_r4282448689
---------
Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-17 21:08:32 +08:00
|
|
|
assert (
|
|
|
|
|
protocol.AGENT_ATTACH,
|
|
|
|
|
{"channel": 1, "task": endpoint.task, "after": 0, "stderr_after": 0},
|
|
|
|
|
) in [a[:2] for a in again.asked]
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
assert pool.status("box").state == "connected"
|
|
|
|
|
await pool.close_all()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class _Gone(_Client):
|
|
|
|
|
"""A connection that answers requests but can carry no notification: it died right after the attach."""
|
|
|
|
|
|
|
|
|
|
async def notify(self, method: str, params: dict[str, Any]) -> None:
|
|
|
|
|
self.said.append((method, params))
|
|
|
|
|
raise AcpConnectionError("the ssh ended", returncode=255, stderr="")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_replay_the_link_cannot_carry_keeps_the_frames_and_every_waiting_endpoint(tmp_path):
|
|
|
|
|
pool = NodePool(backoff_s=())
|
|
|
|
|
client = _live(pool, tmp_path, client=_Gone())
|
|
|
|
|
first = ChannelEndpoint(client, 1, target=_box(), agent="a@box") # type: ignore[arg-type]
|
|
|
|
|
second = ChannelEndpoint(client, 2, target=_box(), agent="b@box") # type: ignore[arg-type]
|
|
|
|
|
for endpoint in (first, second):
|
|
|
|
|
endpoint.detach("the link went")
|
|
|
|
|
await first.write(b"one\n")
|
|
|
|
|
await first.write(b"two\n")
|
|
|
|
|
pool._parked["box"] = [first, second]
|
|
|
|
|
|
|
|
|
|
await pool._reattach("box")
|
|
|
|
|
|
feat(*): read, write and deliver files on a registered machine by address (#922)
## Summary
Third of four changes that let a Raven dispatch work to the machines in
its machine registry. #920 is merged; this branch is opened against main
ahead of #921, so the diff here includes #921's commits as well: review
from "read, write and deliver files on a registered machine by address"
on (the commits before it are #921's, carried here until it merges).
What this does:
- The file tools take a path on a registered machine as it is written
for scp -- `user@host:/path`, or `scp://user@host:port//path` when the
port is not 22 -- and answer it through the machine's daemon
(`raven/agent/tools/remote_files.py`, hooked from `filesystem.py` and
`file_search.py`): `read_file`, `list_dir`, `find` and `grep` run
Raven's own tools there and bring back the answer, so a read of fifty
lines of a large file moves those fifty lines; `write_file` and
`edit_file` write there, with the digest of that agent's own last read
-- the conversation's, or its sub-agent run's -- as the expected
state, so a file changed since it read it is refused and nothing is
written, and another agent's read does not stand in for its own. The
path is resolved against the registry
only, never this computer's ssh config, so one path names one machine
on any computer; one that matches no row is refused, not guessed.
- What is shown back as an address is only what the tool says about
paths -- its errors, its headers and footers, a write's account, a
diff's two header lines; a read's numbered lines, a search's matches
and context, and a diff's body come back as they are, so the model
edits against the bytes the file holds. The shell's refusal of typed
`ssh` to a registered machine (`raven/agent/tools/shell.py`,
`machine_exec.py`) reads the program in any spelling -- a path, a
backslash, and on a Windows host `ssh.exe`, `SSH.EXE` or a whole
`C:\...\ssh.exe`, with the backslashes the shell there keeps and
without the quotes it takes off (a quoted piece and a bare one next to
it making one word), so a quoted destination or port is matched like a
bare one -- and, with the lane on, names the file tools. `scp`, `sftp`
and `rsync` to a registered machine stay what they are on main: a copy
starts nothing there and passes exec's own question like any command,
and the file tools cannot carry a directory, a binary file or a large
one, so the copy stays the route for those. A drive-relative or
extended Windows path (`C:file`, `\\?\C:\...`, UNC) is this
computer's, not a machine nobody registered.
- Permissions (`raven/permissions/`): a path inside the directories
Raven's agents work in on that machine -- its session directory, its
`remoteCwd`, the row's listed `paths` -- is read without asking;
elsewhere a read asks once per machine and directory for the session.
A write there asks wherever it falls, as a write here does, and its
grant is for the one file, apart from any read. The daemon checks the
roots again on the machine, links included.
- `deliver_files` fetches one file by its address, whole and checked,
under a 20 MB cap, into `<workspace>/pulled/<machine>/<key>/<name>`,
the key a digest of the file's path on the machine, so two paths never
share a landing: not two files of one name from two directories, and
not two names this computer's filesystem reads as one (`Report.txt`
and `report.txt`, a trailing dot). The name beside the key is written
as one this computer keeps (a backslash, what Windows reserves in a
name, control characters and the percent sign itself as
percent-codes), and a landing that would not resolve under the
machine's delivery directory is refused before anything is made; a
bigger file is named with its size and read there instead. Only a file
under the directories this Raven's agents work in
on that machine is fetched: the tool asks nothing, and a read some
conversation was allowed is no evidence for this one, so a file
elsewhere is refused and read there instead. Nothing reclaims
`pulled/`: a delivery is opened from its landing for as long as its
record lasts, and a bound by size or age is left out of this change on
purpose.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_remote_file_tools.py
tests/test_permissions_remote_read.py tests/test_deliver_files_tool.py
tests/test_shell_machine_channel.py tests/test_ops_remote_path.py
tests/test_permissions_gate.py tests/test_import_cycle_budget.py
tests/test_node_server.py tests/test_node_client.py
tests/test_subagent_acp_remote.py tests/test_acp_remote.py
tests/test_cli_ops_connection_commands.py`
-- all passed on this tree (703 in the run that covered the first
eight groups, the node's own suite and the real-node cases among them).
Covered: each tool on a remote path answers as its local tool would,
through a stood-in link and a real daemon; a path outside the roots is
refused on both sides; a write with a stale digest is refused and
writes nothing; the address form on what the tool says about paths,
and a read's lines, a grep's matches and a diff's body untouched; the
shell's refusal of typed ssh to a registered machine, by host, by
alias, in a pipeline, as `ssh.exe` and `SSH.EXE`, and with a whole
Windows path or a quoted destination or port when the host is Windows,
a copy with scp, sftp or rsync not refused, and the file tools named
only with the lane on; a fetched name holding the
other platform's separators lands as one name under the machine's
directory and nothing lands elsewhere, and a landing that would
resolve outside fetches nothing; a Windows path form read here with the
lane on; the read
permission inside the roots and the question elsewhere, remembered per
machine and directory, a write asked wherever it falls and its grant
kept per file and apart from a read; a read's digest kept per agent,
so another agent's read lets no stale overwrite through and an agent
that never read the file expects nothing; a delivery fetched whole
under a key of its path (two files of one name, two names a
filesystem reads as one, and the percent-code pair all land apart),
refused over the cap and refused outside the roots whatever was read
before.
- `uv run pytest` on this tree after the merge with main -- 31210
passed,
231 skipped, 2 xfailed, 4 failed, every one of the four in a test that
needs what this machine lacks (`test_rpc_files.py`, the two simulation
cases and the kimi-code case that fail the same way on main here); the
groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Diff coverage of this change against main after the merge: 95.27% (805
of
845 executable changed lines), against the 90% gate. The node suite,
the remote-path tests and the real-node cases among the groups above run
on the merged tree.
- Real machines (two registered Linux machines): fifty lines from the
middle of a 5 MB file read through the daemon in 30 to 35 ms, byte for
byte what this computer reads; list, find and grep answered; a write
and an edit landed as the local tools would, and a write after the file
changed was refused; a path outside the roots refused; a file written
by an agent there fetched back by address. The shell's refusals are
covered by the tests above.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint`,
`ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
User docs follow once the lane is no longer debug-only.
## Risk
Behind the debug lane, as the two changes before it. With it on, a
remote
path reaches the machine only through its daemon, inside the roots each
call names and under the read and write questions above; the shell
cannot
run a command on a registered machine by address, and a copy to or from
it
with scp, sftp or rsync is the plain shell's behind exec's own question,
as
on main. A write refused for a stale
digest writes nothing. A delivery fetches at most 20 MB per file into
the
workspace, and only from the directories the agents work in on the
machine. Rollback is a plain revert.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#920 (merged); depends on #921.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 16:10:18 +08:00
|
|
|
assert [params["line"] for _, params in first._pending] == ["one", "two"], (
|
|
|
|
|
"nothing the link did not carry is dropped"
|
|
|
|
|
)
|
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary
First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.
It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.
After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.
What a daemon is (`raven/node/`):
- This Raven's own code, running as a daemon on a registered machine.
The host's ssh runs a relay there (`python -I -m raven.node --attach
--dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
so nothing is registered as a service and a reboot costs one connect.
It opens no network port; the socket is the registered user's own
(mode 600) in `run/` (0700), or, when that path is too long for a
socket, in a directory of the user's own under the temp directory
(0700, checked before use: a directory of the user's, never a link,
never another user's, narrowed when wider). The relay forwards only to
a socket of the user's and starts no daemon beside anything else; a
daemon whose socket's place holds a thing it cannot remove says so in
one line and leaves rather than listen beside it. ssh login is the
authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
host drives it with `AcpClient` (over an `Endpoint` seam this change
adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
`pull`) run Raven's own file tools there, fenced to the `roots` each
call names, so a remote read comes back exactly as a local one would.
A write may carry `expect`, the digest the file had when it was read,
and is refused with `CHANGED` when it no longer matches. A page read
with its digest, a stat with one, a chunk and a checked write run in a
thread of the daemon's, writers one at a time under a lock, so the
loop that answers the heartbeat and forwards every agent's frames is
not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
the command named is started in the machine user's login environment
(captured once by the daemon itself, under `bash` or `zsh` `-lic`),
in its own process group, under a task id of the host's. Its ACP
frames travel on a numbered channel of the connection that holds it,
and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
files 0600, as the host's own journal is) as they arrive. A connection
that goes leaves the agent running; `agent/attach` on a later
connection replays the record from the frame the host last read and
takes the task over, and a finished task is replayed from its record
alone, even by a daemon that never ran it. `agent/cancel` ends the
whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
records are kept seven days. `node/shutdown` ends agents only under
`--stdio`, the one mode that ties them to the connection (tests, and a
fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
model, reads a config or opens a port; beside Raven's code the machine
needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
one (else uv brings one), loguru and pillow (about 55 MB, more with a
fetched uv or Python), ripgrep-bin when its index has it. A test runs
the node with nothing importable beyond the standard library, its own
code and those two packages, and asks every file call, so the next
import added under `raven/agent/tools` is caught here rather than on
every installed node.
The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.
The host side (`raven/node/client.py`):
- One link per machine, shared by every caller in the process; a link
idle for ten minutes with no agent on it and no call on its way to it
is closed the next time any link is asked for. A heartbeat every 5 s;
one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
waits; a link that comes back attaches every task again from the frame
the host last read, sends what waited and any cancel asked in between.
Only a task the machine no longer knows ends an endpoint, in one
sentence. A link closed on purpose just closes: the daemon keeps its
agents, and `tasks()` on a later link lists them. The listing says
"link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
never its address: not installed (with the install command),
unreachable, login refused, other code, a refused handshake, a call
that ran out of time, or the daemon's own last word.
The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.
Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.
Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
-- 859 passed on this tree; diff coverage of this change's lines from
these groups alone passes the 90% gate. Covered, beside what #901's
tests covered (parity of every file call with the tool's own answer,
the fence, each malformed field, the handshake, the stdio loop, the
install's scripts and one real install on this computer, through uv):
- The socket's directory is the user's own or not used (a file, a
link and another user's directory refused; wider bits narrowed); a
daemon refused its socket's place says so and leaves; the relay
forwards to nothing that is not the user's and starts no daemon
beside it; a task's records are 0700 and 0600; a ping is answered
while a file's digest is taken; a call on its way keeps its link;
the ssh argv reaches the connection as it is and `AcpClient.launch`
takes an argv; the installer fetches uv when the machine has none
(curl or wget) and refuses a machine with neither; a drive-relative
path is this computer's on a Windows host and scp's elsewhere, an
extended or UNC path this computer's anywhere; the node answers
every file call with only its requirements importable.
- A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
hears the exit; a finished task is replayed from its record by a
daemon that never ran it; a record left "running" by a daemon that
died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
the first, the daemon leaving when idle and leaving at once when
another serves the same directory; `--serve` and `--attach` need
`--dir`.
- The link, over a stand-in `ssh` that hands the line to a real login
shell and starts the real relay and daemon: a request made while the
link is down is answered after the link is back, on the same daemon;
a daemon told to stop takes its agents and the turn says so; a link
closed on purpose leaves the agent for the next link to list, attach
and cancel; a heartbeat lost parks the endpoint and the reconnect
attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
waiting endpoint again; held frames are capped; a file fetched whole
and checked, and refused outside the roots.
- The relay in this process: it starts a daemon when none answers,
bridges bytes both ways, reaches the same daemon again without
starting another, and gives up in one sentence when none comes up.
- Cancel ends a child the agent put in a session of its own; the tree
is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
10 kept, 0 broken. The daemon's two test modules skip on win32 before
importing it (fcntl, unix sockets), and the two POSIX teardown tests
lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
this one -- 29217 passed, 121 skipped, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
which fail the same way on main on this machine. The groups above
also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
Linux machines, a scratch registry and `RAVEN_HOME`):
- Install through uv on both: on the machine without uv, uv fetched
with the machine's curl under the node directory, then a virtualenv
of the machine's own Python 3.12, 16 s in all; on the machine whose
home disk was full, under `/tmp`, with a Python uv brought, 11 s.
Both nodes answered the handshake with this code's digest.
- Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
`agent/launch` 13 ms; a request's round trip 12 ms.
- With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
after the kill with the agent's reply, 25 of 25 lines written, the
same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
tests/test_node_install.py tests/test_node_server.py
tests/test_node_client.py tests/test_read_file_image.py` -- 337
passed, 100% diff coverage of that commit's lines): a page is the
lines `splitlines` gives whatever the chunk edge, over every break it
knows; a 120 MiB file pages with every ping answered within 0.25 s; a
read with a slowed digest beside a write of the same file answers with
the old page and the old digest and the write lands after; a waiting
writer goes before new readers; the probe reads `pip.conf` and
`PIP_INDEX_URL` as pip does under a real shell, the plan names the
index and the install hands it to uv; the uv step fetches one pinned
version unmanaged; a node directory that was there keeps its mode and
one the install makes is owner-only. On a registered Linux machine
with the node reinstalled: a page read of a 1 GiB file there beside
pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
sent during a write all answered with a page and a digest of one
version; the probe found the mirror that machine's pip is configured
with; an install into a fresh node directory, forced down the fetched
path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
seven new tests failing on the code before the fix, diff coverage of
that commit's lines past the gate): a link the heartbeat has not yet
seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
being replayed and every one not yet reached, and the link is tried
again; a cancel asked on a dead link, or failing on the new one, stays
wanted until a link delivers it, and a refusal clears it; two installs
on one long node directory get sockets apart; a launch still capturing
the login environment is counted, so a second one at the limit is
refused before the first spawns. On a registered Linux machine, one
script run before and after the fix (the local relay killed with the
heartbeat kept out of the way, then a plain call; a cancel asked in
that gap; eleven launches at once against the limit of eight): before,
the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
there; after, the call took 0.3 s and the agent answered through the
new link, the task ended within the reattach with its record saying
so, eight started and three were refused with eight running, and
nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
the commit and the prospective squash header, `ruff` -- all clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed
The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.
## Risk
Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follows #893; replaces #897 and #901.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
|
|
|
assert pool._parked["box"] == [first, second], "the one being replayed and the one not yet reached both wait"
|
|
|
|
|
assert first.detached and second.detached and first.returncode is None and second.returncode is None
|
|
|
|
|
assert not client.alive and "box" not in pool._nodes
|
|
|
|
|
assert len(client.said) == 1, "the second frame was not tried on a link that had just failed"
|
|
|
|
|
assert "box" in pool._reconnects, "and the link is tried again"
|
|
|
|
|
assert pool.status("box").state == "lost" and pool.status("box").agents == 2
|
|
|
|
|
await pool.close_all()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_cancel_the_link_cannot_carry_is_kept_for_the_next_link(tmp_path):
|
|
|
|
|
dead = _Client(raises=AcpConnectionError("the ssh ended", returncode=255, stderr=""))
|
|
|
|
|
endpoint = ChannelEndpoint(dead, 1, target=_box(), agent="echo@box") # type: ignore[arg-type]
|
|
|
|
|
|
|
|
|
|
await endpoint.end() # the link died; the heartbeat has not said so yet
|
|
|
|
|
|
|
|
|
|
assert endpoint._cancel_wanted and endpoint.returncode is None
|
|
|
|
|
again = _Client()
|
|
|
|
|
await endpoint.reattach(again, 3)
|
|
|
|
|
assert (protocol.AGENT_CANCEL, {"channel": 3}, node_client._CANCEL_TIMEOUT_S) in again.asked
|
|
|
|
|
assert not endpoint._cancel_wanted, "delivered"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_cancel_that_fails_during_the_reattach_is_still_wanted_after_it(tmp_path):
|
|
|
|
|
endpoint = ChannelEndpoint(_Client(), 1, target=_box(), agent="echo@box") # type: ignore[arg-type]
|
|
|
|
|
endpoint.detach("the link went")
|
|
|
|
|
await endpoint.end()
|
|
|
|
|
assert endpoint._cancel_wanted
|
|
|
|
|
dying = _Client(raises=AcpConnectionError("the ssh ended", returncode=255, stderr=""))
|
|
|
|
|
|
|
|
|
|
await endpoint.reattach(dying, 2) # the cancel's request fails on the new link too
|
|
|
|
|
|
|
|
|
|
assert endpoint._cancel_wanted, "not delivered: still wanted"
|
|
|
|
|
again = _Client()
|
|
|
|
|
await endpoint.reattach(again, 4)
|
|
|
|
|
assert (protocol.AGENT_CANCEL, {"channel": 4}, node_client._CANCEL_TIMEOUT_S) in again.asked
|
|
|
|
|
assert not endpoint._cancel_wanted
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_cancel_the_node_refuses_is_not_wanted_any_more(tmp_path):
|
|
|
|
|
refusing = _Client(raises=AcpRemoteError(protocol.AGENT_CANCEL, protocol.UNKNOWN_TASK, "no agent on channel 1"))
|
|
|
|
|
endpoint = ChannelEndpoint(refusing, 1, target=_box(), agent="echo@box") # type: ignore[arg-type]
|
|
|
|
|
|
|
|
|
|
await endpoint.end()
|
|
|
|
|
|
|
|
|
|
assert not endpoint._cancel_wanted, "the node answered: there is nothing there to cancel"
|
feat(*): lend the host's MCP servers to an acp agent on a machine through its node (#947)
## Summary
Adds MCP lending for remote ACP agents on top of the machine and
recovery
services already merged in #920, #922 and #934. The branch includes
current
main and its default-on registered-machine behavior.
What this does: an acp entry on a registered machine is handed the MCP
servers its row grants, the way an entry on this computer is, and the
servers themselves stay here with their definitions and keys.
- The agent on the machine runs the node's own bridge as a stdio MCP
server (`python -I -m raven.node --mcp --dir <node dir> --task <task>
--lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte
pumps around a connection to the daemon's socket, no MCP parsed, no
credential held, the standard library only.
- The daemon carries the bridge's frames as a *stream* to the host
holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6,
`raven/node/protocol.py`, `raven/node/server.py`). The daemon tells
which
task a bridge is by the process that connected -- the peer pid off the
socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the
parent chain to the agent the daemon started, which the bridge is under
and no other task's process is; the kernel supplies that pid and a peer
cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the
task it names must be the one the connection is a process of. Two tasks
on
one machine run as one user, so no name or file a bridge could present
would tell them apart; the connecting process does. One stream per
bridge, at most 16 per task; a frame
travels in pieces of at most 256 KiB characters, the last marked
(`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the
daemon relays and keeps nothing. A stream ends with either end, the
host's link or the task, and the other end is told.
- The host opens the endpoints for a machine row exactly as for a row
here
(`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in
the host's sandbox, HTTP auth added here), binds them to the machine,
dispatch and acquired agent task
(`NodePool.lend`/`unlend`) before handing out bridge stanzas. On
`mcp/open`
the host checks the daemon-authenticated task against that binding
before
connecting; it rechecks the grant and link after the socket opens. An
unlend, replacement grant or replaced link cannot authorize that pending
connection. The host then pumps the frames both ways
(`raven/node/client.py`).
`node/hello` names the bridge; a node serving without a daemon
(`--stdio`)
names none, and the dispatch then goes without MCP and says so.
- The reply closes with the servers lent and the machine's node, or with
why none were. Remote lending requires Unix-socket support on the host:
on Windows it is withheld before endpoint setup, the task continues, and
the reply explains why. The roster's "injectable MCP" matches that
support
(`session_mcp_delivered`). The cli lane on a machine still
withholds its servers (the next change); the openai lane has no tools.
Not in this change: the cli lane's MCP file on the machine; a stream
that
survives the link going (a bridge sees its server end, the agent reports
the call failed, and the next dispatch works again); MCP for a run
picked
up after this computer restarted (#934): it reports its calls fail.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Follow-up fix 05eaba67a, after the c9de53947 review:
- The nine tests the Windows review named now pin the host fact through
one
fixture, `unix_socket_host` (`tests/conftest.py`), which sets
`raven.mcp.bridge.unix_sockets_available` to true and re-points every
`raven.*` module that imported it by name; their bodies are unchanged.
- Found complete, not just the nine: the whole suite was run with
`asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted
before collection (a pytest plugin, controller and xdist workers), at
the
merge base `6b938a426` and at `c9de53947`, and the two failure sets
compared. 17 node ids failed at the head and not at the base: the nine,
and eight this change added that run the transport for real (the bridge
module, and two lending tests through the stand-in machine). Those eight
are guarded by host capability instead of pinned: the bridge module
skips
when `sys.platform == "win32" or not unix_sockets_available()`, the two
lending tests carry the existing `skipif(not unix_sockets_available())`.
After the fix the same run fails the same 66 node ids at the head as at
the base: nothing this change adds or touches fails on a host without
unix sockets.
- A bridge whose daemon side goes ends as streams end. The host can
close a
stream while a frame of the agent's is on its way; the daemon writes
`mcp/close` and hangs up, and the frame meets the closed socket, or the
read side is reset first. That exited 1 with `BrokenPipeError` or
`ConnectionResetError`; a connection-level end is now exit 0 with
nothing
said, a frame past the cap still exits 1 with its reason, both pumps are
awaited before the socket closes, and a daemon gone before the stream
opens is one sentence, not a traceback. Four deterministic cases with a
stand-in connection fail on `c9de53947` and pass here.
- The bridge test's stand-in agent outlives its bridge, as an agent
outlives
one of its MCP servers, so the later parts of
`test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins`
no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80
here.
- The focused set (node, MCP, remote, manager and roster files) -- 935
passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
simulation cases, which fail the same way on main on this computer);
diff coverage against main 97.67% (420/430). Real machine, node
reinstalled with this tree in a directory of the check's own: a stdlib
ACP
agent's bridge reached a host-held server and 1 MiB and 8 MiB results
crossed whole, bridge exit 0; nothing was left on the machine.
Follow-up fix c9de53947, after the same-user token finding:
- The daemon no longer uses a token. `agent/launch` mints and writes
none,
`mcp.token` and the `hmac` check are gone, and `Daemon.open_stream`
identifies the bridge by the connecting process instead (`_peer_pid` +
`Daemon.task_of`). The same-user path the reviewer kept open -- a peer
reading another task's `mcp.token` and opening its stream -- is closed,
because identity is the connection the kernel reports, not a readable
secret.
- Tests, each failing on the unfixed tree first: a bridge placed under
task
B that names task A is refused (`the connection is a process of task
'agent@box-2', not 'agent@box-1'`), as are a process under no task, an
unplaced connection, and a task the daemon does not run
(`tests/test_node_server.py`); a real-OS test launches an agent, takes
its
real child, and asserts the child resolves to the task while the test's
own
process resolves to none (runs on Linux in CI); `_peer_pid` is checked
against a real socketpair; the in-process and subprocess bridge tests
now
identify by the connecting process, and a bridge this test spawns
itself,
or another task's agent spawns, never opens the stream.
- Windows withholding now covers a row on this computer too, not only a
machine row: `session_mcp_delivered` drops the `machine and` guard and
answers no whenever `unix_sockets_available()` is false, so a local acp
row is no longer advertised as injectable and no longer raises at
dispatch
on a host without the transport; `_mcp_endpoints` yields nothing before
opening an endpoint and `_mcp_note` explains it. The two tests the
Windows
review named now guard on `unix_sockets_available()`, and the
withholding
test is parameterized over a machine row and a local row.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over the node/MCP/remote suite -- 97.63% changed-line coverage
(412/422).
`ruff`, `ty` on the changed modules, and the import-cycle budget pass.
- Real machine (ravenx2, this head, node reinstalled, digest
fdbe132e76f0):
a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB
and
an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the
connecting-process identity admits the legitimate bridge end to end on
real Linux; the cross-task refusal on Linux is the real-OS test above,
which CI runs there.
Earlier review fixes (4dc5b3e1f, the task-to-grant binding). Its
host-side
binding of each dispatch's endpoints to the acquired agent task is kept;
the
daemon-side token this run verified was since replaced by the
connecting-process
identity above (c9de53947), so the token-named tests and reproduction
below no
longer describe the current tree.
- `uv run --no-sync pytest tests/test_node_client.py
tests/test_node_server.py tests/test_node_bridge.py
tests/test_node_relay.py tests/test_node_bundle.py
tests/test_node_install.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py
tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict
--cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report=
--tb=short` -- 1495 passed and one minimal-node stdio response test
failed;
strict idle accounting also flagged the assembly test. Both cases pass
together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or
timing limits were weakened. The new authorization, revocation and host
compatibility tests all pass.
- The five focused cases failed on the unfixed implementation first: a
different authenticated task selecting another grant, a stream with no
task, each missing Unix socket API, and an astral Unicode chunk over the
node's byte limit. Regressions also cover revocation, grant replacement
and link replacement while an endpoint connection is opening, and a real
remote turn continuing when both host Unix socket APIs are absent.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 97.94% (381/389 changed executable lines); this fix alone is 100%
(40/40). Python lint, type, dependency
and import checks pass; all 10 import contracts hold. The scope checks
for commit messages, source language and file sizes pass. Both RPC
generators pass `--check` after integrating current main.
- A real Linux node with two concurrent tasks and a host loopback
endpoint:
before, the attacker's own task token plus the victim's public grant
names returns the victim endpoint's marker; after, it returns no data
and the endpoint is never reached. Its own grant works, unknown and
revoked grants fail, and the same attack remains denied after the relay
is killed and an ordinary ping reattaches both tasks. A full chunk of
262144 astral characters was lost/corrupted before and arrives intact
after. The final documented source tree was reinstalled and checked
again.
- Real ACP dispatch through the node's bridge reaches a host-held stdio
MCP tool and returns its marker; its grant is removed when the turn
ends.
With the host Unix socket APIs removed to simulate Windows, the remote
agent still completes, receives no MCP, explains the host limitation,
and the host MCP ledger records no extra call. This is not a native
Windows run. Dedicated test tasks, host endpoints and bridge processes
are cleaned up.
- CONTEXT.md, module/field docstrings, both collaboration guides, the
external design notes and this description were checked against the
implementation. Main's no-debug-switch behavior and full-turn recovery
documentation are retained.
Full combined-tree verification: `uv run --no-sync pytest -n 4
--tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure
reproduces on current main in a separate checkout: host CLI color
rendering,
LibreOffice detection and the two simulation cases. No other test fails.
The minimal-node response test that failed in the focused parallel
selection
passes in this full run. Diff coverage: 97.94% (381/389 executable
changed lines).
Earlier author verification:
- `uv run pytest tests/test_node_bridge.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_bundle.py tests/test_node_install.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py`
-- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge
and a stdio node naming none, the token written owner-only beside the
record, a stream opened with the task's token and refused with another
or for a task not running, refused with no host holding the task,
refused past 16 per task, one per bridge connection, the pieces of a
frame crossing both ways as they are and a stranger carrying nothing,
and every way a stream ends (the bridge closing, the host closing, the
bridge's connection ending, the host's link ending, the agent ending)
with the other end told; the bridge as a process against a real daemon
over its socket: a frame to the host and one back whole, a large frame
each way in pieces, the agent closing its stdin ending the stream and
the bridge leaving clean, the host closing the stream ending the bridge,
a wrong token and a missing daemon each refused on stderr with exit 1;
the stand-in machine with the real daemon and the real bridge: an acp
stub there that connects to its `mcpServers` and calls a tool reaches a
stdio server here and answers with its result, the stanza handed to it
naming the node's bridge, the task, the dispatch and the server and
nothing of the server's definition or socket, the ledger here showing
one call, no bridge outliving the dispatch; the reply's note and the
roster predicate for a machine row.
- `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed,
4 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`,
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`
and
`test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`,
which fail
the same way on main on this machine.
- Real machine (a registered Linux machine with Claude Code signed in,
the node reinstalled with the code before and after):
A28, before (the
code of #934's head and its node): the turn closed with "MCP servers
'probe' were not lent", Claude Code said no tool of that name was
connected, and the server here saw no call; after: `PROBE_OK:r2-lend`
in 9.4 s, the server here recorded the call, and the turn closed with
"lent to acp agent ... through the Raven node on machine ...". A34:
after
the turn the machine's process list shows no bridge, a grep of its node
directory finds nothing of the server's command or of this computer's
socket directory, and no endpoint socket is left here. A29: with the
tool switched off here, Claude Code answered UNAVAILABLE and the server
saw no call. A30: a 40 s tool call under way, the bridge visible in the
machine's process list, then cancelled: 3 s later no bridge there and
no socket here. A31: the relay ssh killed during a 20 s call: the turn
ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next
dispatch reached the server again in 7.6 s. A32: five calls in one turn
all reached the server and came back; an 8 MiB answer: the server here
produced it, and a plain ACP agent on the machine (a stdlib stub that
speaks the stanza's bridge directly) received it whole through the node
-- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude
Code itself closed the server on that answer and reported the call
failed: the agent's own cap on a tool result, not the link's.
- `ruff`, the commit-message and PR-body checks -- clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese collaboration guides describe lending,
task-bound authorization,
Windows withholding, and the existing CLI/restart limitations.
## Risk
Registered-machine support is enabled by configuration, without the
removed
trial environment switch. An ACP entry on a machine receives its granted
MCP servers on a Unix-socket-capable host; the host's normal MCP policy
still
applies. This intentionally exposes those tools to the selected remote
agent.
The server definition, credentials and endpoint paths remain on the
host.
The daemon identifies a bridge's task by the connecting process (the
peer
pid the kernel reports, walked to the agent it started), not by any
token or
name the bridge presents, so a peer task sharing the OS account cannot
open
another task's stream; the host then authorizes that task against the
dispatch/server grant. A system that reports no peer pid opens no
stream.
On Windows hosts remote lending is withheld explicitly, without aborting
the
remote task. Protocol 6 remains the MCP wire version; host and node must
also
have the same code digest, so updated nodes must be reinstalled.
Rollback is
a revert; the machine then carries no stream and receives no bridge
stanza.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#934 (merged); #920 and #922 (merged).
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-15 15:37:50 +08:00
|
|
|
|
|
|
|
|
|
fix(*): machine runs outlive a killed daemon, a refused failure announce, a dead-link cancel (#957)
## Summary
Seven edges of a run on a registered machine, left by the review of #934
and by timing probes on the real machine. Each is a fix the host or the
node makes on its own; nothing changes for an entry without `machine`.
- **A Raven node killed outright no longer leaves agents behind or a
bare `exit -1`.** The next daemon brings every record left at `running`
to `lost` at its start (`Daemon.recover`), ends the agent when its pid
is still the process it was started as (group leader, the recorded
command, and the start mark the launch recorded: Linux's `/proc` start
tick, elsewhere `ps -o lstart`), and writes why; a record without the
mark, or a pid whose mark differs, is left alone and recorded as gone.
`agent/exit` and the attach answer carry that `reason`; the host hands
it on in place of the exit code (`ChannelEndpoint.exited(rc, reason)`),
so a cli run says "the Raven node on this machine was restarted while
the agent ran, and the agent is gone" and a picked-up acp run says the
same. On Linux the agent is also tied to the daemon's life
(`PR_SET_PDEATHSIG`, a SIGTERM). Protocol number unchanged: the field is
optional and old nodes never send it.
- **A failed remote run's error survives a draining-refused announce.**
The manager's two error branches read whether the announce was taken, as
the reply branch does (`announced = taken is not False`), and the acp
backend keeps the record of a prompt that went out and failed, the cli
backend that of a timed-out command, for the caller that reports it. The
next start reads how the task ended off the machine and announces the
error.
- **A cli run picked up after a restart binds its instance to the
session id in its output** (`_bind_picked_up`), as the dispatch's
deferred commit would have; with no id in the output the reply carries
the same "not resumable" warning the dispatch would. A `provisioned` id
or a stateless run binds nothing.
- **A cancel while the machine's link is known dead returns at once.**
`_cancel_turn` reads `ChannelEndpoint.offline` (parked, or the link not
alive), marks the session unsettled and leaves the held `session/cancel`
for the next link, instead of waiting the settle budget for a settlement
that cannot arrive. In the gap before the heartbeat notices nothing
changes.
- **An openai row's tunnel gives up a taken local port for another.**
The port is checked again just before ssh starts (`_still_free`), and
ssh's own "Address already in use" / "Could not request local
forwarding" ends in another try (`_TUNNEL_TRIES` = 3) rather than in a
request sent to whoever took the port; each attempt reads only what ssh
wrote in its own run of the log.
- **A run record that is not JSON, or not UTF-8, is set aside once**
(`<name>.json.unreadable-<ts>`), named in the log that once, instead of
being warned about at every start; a record a later version wrote is
left alone as before. (The first revision read the record as text and
let a `UnicodeDecodeError` out of the whole scan, found by the Windows
review; the record is read as bytes now and decoded by `json.loads`, so
a decoding failure is set aside like a JSON one, and one bad record
never stops the others from being read.)
- **A path written as remote that names no registered machine** is
refused in one sentence for `write_file` and `edit_file` too, never
taken for a relative path here. This has been the case since the debug
switch went (#934); a regression test pins it.
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Every fix has a test that failed on the old code first.
- `.venv/bin/python -m pytest -n 6 -q tests/test_node_server.py
tests/test_node_client.py tests/test_remote_runs.py
tests/test_subagent_manager.py tests/test_subagent_cli_remote.py
tests/test_remote_file_tools.py tests/test_subagent_acp_remote.py
tests/test_resume_remote.py tests/test_remote_resume.py
tests/test_subagent_acp.py` - all green on macOS (the Linux-only pact
test skipped there); the last run of the four fast files 331 passed, cli
remote 52, remote file tools 24, the acp remote recovery tests 3
- Full suite: 31577 passed, 234 skipped, 2 xfailed in 6 min 14 s; the 3
failures are the ones `main` has on this machine (a host without
LibreOffice, two simulation records). Changed-line coverage: 98.18% of
165 executable changed lines (`scripts/coverage_gate.py diff --base-ref
origin/main`); the 3 uncovered lines are the Linux-only `prctl` import
block, which CI's Linux shards run.
- `ruff check` / `ruff format --check`, `ty check` on the changed
modules, `lint-imports` (10 contracts kept),
`scripts/check_commit_file.py`, `scripts/check_source_language.py`,
`scripts/check_large_files.py`: clean.
- Real machine (ravenx2, an isolated registry copy and Raven home per
tree; the node of each tree installed under its own `node_dir`; the same
probe script on this branch and on `main`):
| Probe | `main` | this branch |
|---|---|---|
| P1 `kill -9` the daemon under a running cli turn | 4.9 s after the
kill: `CLI agent 'x' exited -1`; the loop the daemon had started was
still running (1 left) | 5.1 s: `CLI agent 'x': the Raven node on this
machine was restarted while the agent ran, and the agent is gone`; 0
left; a new daemon pid |
| P2 an acp prompt that fails after it went out, `reported_by_caller` |
the prompt failed (`exit 5`); no record left | record kept; the next
start announced the error with `exit 5` and removed the record |
| P3 cancel while the link is known dead; the cancel on the next link |
the cancel returned after 2.35-2.46 s, held until the link was back (a 2
s backoff in the probe; up to the 5 s budget in production); it landed
on the next link (task frames 3->4) | 0.00 s; landed the same way (3->4
within 0.1 s of the restore) |
| P4 the chosen local tunnel port is taken by a listener | the tunnel
used the taken port; the request timed out against the listener that
held it | the next port; HTTP 200 through the machine |
| P6 a derived-id cli create left running, picked up by the next start |
picked up ok, no binding (`commits=[]`) | picked up ok, the handle bound
to `edge-123` |
After the merge of `main` (which brought #947's MCP streams, #953's
host-death sweep and #963's install fixes into the same files): the
related groups (node server/client/bridge, remote runs, resume, cli
remote, acp remote, manager, remote file tools, acp client, import-cycle
budget) 1006 passed; diff coverage 97.30% of 185 changed lines (the 5
uncovered are the Linux-only `prctl` lines, run by CI's Linux shards);
the real-machine probes re-run on the merged tree: 8/8 PASS (P1 5.2 s, 0
orphans; P2 record kept and the error announced; P3 0.00 s and the
cancel landed on the next link; P4 the next port, HTTP 200; P6 bound to
`edge-123`).
Docs checked against the code before pushing: CONTEXT.md's Raven node
entry (records of failures, set-aside records, the daemon's recovery and
the `reason`), the module docstrings of `remote.tunnel`,
`remote_runs.pending`, `Daemon.recover`, `_still_ours`, `_death_pact`,
`ChannelEndpoint.exited/offline`, `_bind_picked_up`,
`_track_on_machine`, the `agent/exit` comment in `protocol.py`, and the
CHANGELOG entry.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
- [x] Security impact considered - the daemon ends only a pid that is
still the process it started: group leader of its own group, running the
recorded command, and started when the record says (`born`, written at
the launch); a reused pid fails the last mark even for the same program
and is left alone, the record then saying the agent is gone. No new
network surface; the `reason` is the node's fixed sentence, never user
or agent text.
- [x] Backward compatibility considered - `agent/exit`'s `reason` is
optional, protocol number unchanged; a host on this branch reads an old
node's exits as before, an old host ignores the field. Records written
before this branch read unchanged.
- [x] Rollback path is clear for risky changes - a plain revert; nothing
is migrated.
## Related Issues
Follow-up to #934 (the two non-blocking notes of its review) and to the
timing probes run on #921/#922/#934.
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-16 10:09:34 +08:00
|
|
|
async def test_an_exit_the_node_explains_reads_as_its_reason_and_the_lost_code():
|
|
|
|
|
"""A daemon restarted under the agent says so on the exit: the endpoint hands the sentence on in place of a bare code."""
|
|
|
|
|
reason = "the Raven node on this machine was restarted while the agent ran, and the agent is gone"
|
|
|
|
|
endpoint = ChannelEndpoint(_Client(), 1, target=_box(), agent="echo@box") # type: ignore[arg-type]
|
|
|
|
|
endpoint.exited(node_client.LOST_RC, reason)
|
|
|
|
|
|
|
|
|
|
assert await endpoint.wait() == node_client.LOST_RC and endpoint.ended() == reason
|
|
|
|
|
|
|
|
|
|
plain = ChannelEndpoint(_Client(), 2, target=_box(), agent="echo@box") # type: ignore[arg-type]
|
|
|
|
|
plain.exited(0, "")
|
|
|
|
|
assert await plain.wait() == 0 and plain.ended() is None, "no reason given: a plain exit, as before"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_an_endpoint_is_offline_once_parked_or_once_its_link_is_dead():
|
|
|
|
|
endpoint = ChannelEndpoint(_Client(), 1, target=_box(), agent="echo@box") # type: ignore[arg-type]
|
|
|
|
|
assert endpoint.offline is False
|
|
|
|
|
endpoint.detach("the link went")
|
|
|
|
|
assert endpoint.offline is True
|
|
|
|
|
|
|
|
|
|
dead = _Client()
|
|
|
|
|
other = ChannelEndpoint(dead, 2, target=_box(), agent="echo@box") # type: ignore[arg-type]
|
|
|
|
|
dead.alive = False
|
|
|
|
|
assert other.offline is True, "gone and not yet said so by the heartbeat: nothing sent now arrives"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_cancel_while_the_link_is_down_does_not_wait_the_settle_budget(monkeypatch):
|
|
|
|
|
"""The cancel rides with the parked endpoint to the next link; the caller is not held for a settlement that cannot come."""
|
|
|
|
|
from raven.acp_client import client as client_mod
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(client_mod, "_CANCEL_SETTLE_S", 30.0)
|
|
|
|
|
endpoint = ChannelEndpoint(_Client(), 1, target=_box(), agent="echo@box") # type: ignore[arg-type]
|
|
|
|
|
client = AcpClient.over(endpoint, name="echo@box")
|
|
|
|
|
try:
|
|
|
|
|
endpoint.detach("the link went")
|
|
|
|
|
future: asyncio.Future[Any] = asyncio.get_running_loop().create_future()
|
|
|
|
|
started = time.monotonic()
|
|
|
|
|
await asyncio.wait_for(client._cancel_turn("s1", future), timeout=5)
|
|
|
|
|
|
|
|
|
|
assert time.monotonic() - started < 1.0, "no settle wait on a link known to be down"
|
|
|
|
|
assert client.take_unsettled_cancel("s1") is True, "the turn may still run on the machine: not reused"
|
|
|
|
|
held = [params.get("line", "") for method, params in endpoint._pending if method == protocol.AGENT_FRAME]
|
|
|
|
|
assert any("session/cancel" in line for line in held), "the cancel itself waits for the next link"
|
|
|
|
|
finally:
|
|
|
|
|
await client.close()
|
|
|
|
|
|
|
|
|
|
|
feat(*): lend the host's MCP servers to an acp agent on a machine through its node (#947)
## Summary
Adds MCP lending for remote ACP agents on top of the machine and
recovery
services already merged in #920, #922 and #934. The branch includes
current
main and its default-on registered-machine behavior.
What this does: an acp entry on a registered machine is handed the MCP
servers its row grants, the way an entry on this computer is, and the
servers themselves stay here with their definitions and keys.
- The agent on the machine runs the node's own bridge as a stdio MCP
server (`python -I -m raven.node --mcp --dir <node dir> --task <task>
--lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte
pumps around a connection to the daemon's socket, no MCP parsed, no
credential held, the standard library only.
- The daemon carries the bridge's frames as a *stream* to the host
holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6,
`raven/node/protocol.py`, `raven/node/server.py`). The daemon tells
which
task a bridge is by the process that connected -- the peer pid off the
socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the
parent chain to the agent the daemon started, which the bridge is under
and no other task's process is; the kernel supplies that pid and a peer
cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the
task it names must be the one the connection is a process of. Two tasks
on
one machine run as one user, so no name or file a bridge could present
would tell them apart; the connecting process does. One stream per
bridge, at most 16 per task; a frame
travels in pieces of at most 256 KiB characters, the last marked
(`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the
daemon relays and keeps nothing. A stream ends with either end, the
host's link or the task, and the other end is told.
- The host opens the endpoints for a machine row exactly as for a row
here
(`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in
the host's sandbox, HTTP auth added here), binds them to the machine,
dispatch and acquired agent task
(`NodePool.lend`/`unlend`) before handing out bridge stanzas. On
`mcp/open`
the host checks the daemon-authenticated task against that binding
before
connecting; it rechecks the grant and link after the socket opens. An
unlend, replacement grant or replaced link cannot authorize that pending
connection. The host then pumps the frames both ways
(`raven/node/client.py`).
`node/hello` names the bridge; a node serving without a daemon
(`--stdio`)
names none, and the dispatch then goes without MCP and says so.
- The reply closes with the servers lent and the machine's node, or with
why none were. Remote lending requires Unix-socket support on the host:
on Windows it is withheld before endpoint setup, the task continues, and
the reply explains why. The roster's "injectable MCP" matches that
support
(`session_mcp_delivered`). The cli lane on a machine still
withholds its servers (the next change); the openai lane has no tools.
Not in this change: the cli lane's MCP file on the machine; a stream
that
survives the link going (a bridge sees its server end, the agent reports
the call failed, and the next dispatch works again); MCP for a run
picked
up after this computer restarted (#934): it reports its calls fail.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Follow-up fix 05eaba67a, after the c9de53947 review:
- The nine tests the Windows review named now pin the host fact through
one
fixture, `unix_socket_host` (`tests/conftest.py`), which sets
`raven.mcp.bridge.unix_sockets_available` to true and re-points every
`raven.*` module that imported it by name; their bodies are unchanged.
- Found complete, not just the nine: the whole suite was run with
`asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted
before collection (a pytest plugin, controller and xdist workers), at
the
merge base `6b938a426` and at `c9de53947`, and the two failure sets
compared. 17 node ids failed at the head and not at the base: the nine,
and eight this change added that run the transport for real (the bridge
module, and two lending tests through the stand-in machine). Those eight
are guarded by host capability instead of pinned: the bridge module
skips
when `sys.platform == "win32" or not unix_sockets_available()`, the two
lending tests carry the existing `skipif(not unix_sockets_available())`.
After the fix the same run fails the same 66 node ids at the head as at
the base: nothing this change adds or touches fails on a host without
unix sockets.
- A bridge whose daemon side goes ends as streams end. The host can
close a
stream while a frame of the agent's is on its way; the daemon writes
`mcp/close` and hangs up, and the frame meets the closed socket, or the
read side is reset first. That exited 1 with `BrokenPipeError` or
`ConnectionResetError`; a connection-level end is now exit 0 with
nothing
said, a frame past the cap still exits 1 with its reason, both pumps are
awaited before the socket closes, and a daemon gone before the stream
opens is one sentence, not a traceback. Four deterministic cases with a
stand-in connection fail on `c9de53947` and pass here.
- The bridge test's stand-in agent outlives its bridge, as an agent
outlives
one of its MCP servers, so the later parts of
`test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins`
no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80
here.
- The focused set (node, MCP, remote, manager and roster files) -- 935
passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
simulation cases, which fail the same way on main on this computer);
diff coverage against main 97.67% (420/430). Real machine, node
reinstalled with this tree in a directory of the check's own: a stdlib
ACP
agent's bridge reached a host-held server and 1 MiB and 8 MiB results
crossed whole, bridge exit 0; nothing was left on the machine.
Follow-up fix c9de53947, after the same-user token finding:
- The daemon no longer uses a token. `agent/launch` mints and writes
none,
`mcp.token` and the `hmac` check are gone, and `Daemon.open_stream`
identifies the bridge by the connecting process instead (`_peer_pid` +
`Daemon.task_of`). The same-user path the reviewer kept open -- a peer
reading another task's `mcp.token` and opening its stream -- is closed,
because identity is the connection the kernel reports, not a readable
secret.
- Tests, each failing on the unfixed tree first: a bridge placed under
task
B that names task A is refused (`the connection is a process of task
'agent@box-2', not 'agent@box-1'`), as are a process under no task, an
unplaced connection, and a task the daemon does not run
(`tests/test_node_server.py`); a real-OS test launches an agent, takes
its
real child, and asserts the child resolves to the task while the test's
own
process resolves to none (runs on Linux in CI); `_peer_pid` is checked
against a real socketpair; the in-process and subprocess bridge tests
now
identify by the connecting process, and a bridge this test spawns
itself,
or another task's agent spawns, never opens the stream.
- Windows withholding now covers a row on this computer too, not only a
machine row: `session_mcp_delivered` drops the `machine and` guard and
answers no whenever `unix_sockets_available()` is false, so a local acp
row is no longer advertised as injectable and no longer raises at
dispatch
on a host without the transport; `_mcp_endpoints` yields nothing before
opening an endpoint and `_mcp_note` explains it. The two tests the
Windows
review named now guard on `unix_sockets_available()`, and the
withholding
test is parameterized over a machine row and a local row.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over the node/MCP/remote suite -- 97.63% changed-line coverage
(412/422).
`ruff`, `ty` on the changed modules, and the import-cycle budget pass.
- Real machine (ravenx2, this head, node reinstalled, digest
fdbe132e76f0):
a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB
and
an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the
connecting-process identity admits the legitimate bridge end to end on
real Linux; the cross-task refusal on Linux is the real-OS test above,
which CI runs there.
Earlier review fixes (4dc5b3e1f, the task-to-grant binding). Its
host-side
binding of each dispatch's endpoints to the acquired agent task is kept;
the
daemon-side token this run verified was since replaced by the
connecting-process
identity above (c9de53947), so the token-named tests and reproduction
below no
longer describe the current tree.
- `uv run --no-sync pytest tests/test_node_client.py
tests/test_node_server.py tests/test_node_bridge.py
tests/test_node_relay.py tests/test_node_bundle.py
tests/test_node_install.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py
tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict
--cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report=
--tb=short` -- 1495 passed and one minimal-node stdio response test
failed;
strict idle accounting also flagged the assembly test. Both cases pass
together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or
timing limits were weakened. The new authorization, revocation and host
compatibility tests all pass.
- The five focused cases failed on the unfixed implementation first: a
different authenticated task selecting another grant, a stream with no
task, each missing Unix socket API, and an astral Unicode chunk over the
node's byte limit. Regressions also cover revocation, grant replacement
and link replacement while an endpoint connection is opening, and a real
remote turn continuing when both host Unix socket APIs are absent.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 97.94% (381/389 changed executable lines); this fix alone is 100%
(40/40). Python lint, type, dependency
and import checks pass; all 10 import contracts hold. The scope checks
for commit messages, source language and file sizes pass. Both RPC
generators pass `--check` after integrating current main.
- A real Linux node with two concurrent tasks and a host loopback
endpoint:
before, the attacker's own task token plus the victim's public grant
names returns the victim endpoint's marker; after, it returns no data
and the endpoint is never reached. Its own grant works, unknown and
revoked grants fail, and the same attack remains denied after the relay
is killed and an ordinary ping reattaches both tasks. A full chunk of
262144 astral characters was lost/corrupted before and arrives intact
after. The final documented source tree was reinstalled and checked
again.
- Real ACP dispatch through the node's bridge reaches a host-held stdio
MCP tool and returns its marker; its grant is removed when the turn
ends.
With the host Unix socket APIs removed to simulate Windows, the remote
agent still completes, receives no MCP, explains the host limitation,
and the host MCP ledger records no extra call. This is not a native
Windows run. Dedicated test tasks, host endpoints and bridge processes
are cleaned up.
- CONTEXT.md, module/field docstrings, both collaboration guides, the
external design notes and this description were checked against the
implementation. Main's no-debug-switch behavior and full-turn recovery
documentation are retained.
Full combined-tree verification: `uv run --no-sync pytest -n 4
--tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure
reproduces on current main in a separate checkout: host CLI color
rendering,
LibreOffice detection and the two simulation cases. No other test fails.
The minimal-node response test that failed in the focused parallel
selection
passes in this full run. Diff coverage: 97.94% (381/389 executable
changed lines).
Earlier author verification:
- `uv run pytest tests/test_node_bridge.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_bundle.py tests/test_node_install.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py`
-- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge
and a stdio node naming none, the token written owner-only beside the
record, a stream opened with the task's token and refused with another
or for a task not running, refused with no host holding the task,
refused past 16 per task, one per bridge connection, the pieces of a
frame crossing both ways as they are and a stranger carrying nothing,
and every way a stream ends (the bridge closing, the host closing, the
bridge's connection ending, the host's link ending, the agent ending)
with the other end told; the bridge as a process against a real daemon
over its socket: a frame to the host and one back whole, a large frame
each way in pieces, the agent closing its stdin ending the stream and
the bridge leaving clean, the host closing the stream ending the bridge,
a wrong token and a missing daemon each refused on stderr with exit 1;
the stand-in machine with the real daemon and the real bridge: an acp
stub there that connects to its `mcpServers` and calls a tool reaches a
stdio server here and answers with its result, the stanza handed to it
naming the node's bridge, the task, the dispatch and the server and
nothing of the server's definition or socket, the ledger here showing
one call, no bridge outliving the dispatch; the reply's note and the
roster predicate for a machine row.
- `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed,
4 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`,
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`
and
`test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`,
which fail
the same way on main on this machine.
- Real machine (a registered Linux machine with Claude Code signed in,
the node reinstalled with the code before and after):
A28, before (the
code of #934's head and its node): the turn closed with "MCP servers
'probe' were not lent", Claude Code said no tool of that name was
connected, and the server here saw no call; after: `PROBE_OK:r2-lend`
in 9.4 s, the server here recorded the call, and the turn closed with
"lent to acp agent ... through the Raven node on machine ...". A34:
after
the turn the machine's process list shows no bridge, a grep of its node
directory finds nothing of the server's command or of this computer's
socket directory, and no endpoint socket is left here. A29: with the
tool switched off here, Claude Code answered UNAVAILABLE and the server
saw no call. A30: a 40 s tool call under way, the bridge visible in the
machine's process list, then cancelled: 3 s later no bridge there and
no socket here. A31: the relay ssh killed during a 20 s call: the turn
ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next
dispatch reached the server again in 7.6 s. A32: five calls in one turn
all reached the server and came back; an 8 MiB answer: the server here
produced it, and a plain ACP agent on the machine (a stdlib stub that
speaks the stanza's bridge directly) received it whole through the node
-- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude
Code itself closed the server on that answer and reported the call
failed: the agent's own cap on a tool result, not the link's.
- `ruff`, the commit-message and PR-body checks -- clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese collaboration guides describe lending,
task-bound authorization,
Windows withholding, and the existing CLI/restart limitations.
## Risk
Registered-machine support is enabled by configuration, without the
removed
trial environment switch. An ACP entry on a machine receives its granted
MCP servers on a Unix-socket-capable host; the host's normal MCP policy
still
applies. This intentionally exposes those tools to the selected remote
agent.
The server definition, credentials and endpoint paths remain on the
host.
The daemon identifies a bridge's task by the connecting process (the
peer
pid the kernel reports, walked to the agent it started), not by any
token or
name the bridge presents, so a peer task sharing the OS account cannot
open
another task's stream; the host then authorizes that task against the
dispatch/server grant. A system that reports no peer pid opens no
stream.
On Windows hosts remote lending is withheld explicitly, without aborting
the
remote task. Protocol 6 remains the MCP wire version; host and node must
also
have the same code digest, so updated nodes must be reinstalled.
Rollback is
a revert; the machine then carries no stream and receives no bridge
stanza.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#934 (merged); #920 and #922 (merged).
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-15 15:37:50 +08:00
|
|
|
# ---- MCP streams lent across the link ------------------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def _endpoint(name: str = "pg") -> tuple[str, list[bytes], Any]:
|
feat: enable MCP lending on Windows hosts (#972)
## Summary
Windows hosts can lend granted MCP servers to local and machine-based
ACP agents through authenticated loopback endpoints. Hosts with Unix
sockets retain that transport. Server definitions, upstream credentials,
sandbox policy and disabled-tool enforcement stay on the host.
- Share one connector between the local bridge and node client. Bind TCP
endpoints only to 127.0.0.1 and authenticate with a fresh per-endpoint
token before opening an upstream.
- Read inherited Windows stdin pipes through a daemon worker with
backpressure. Acknowledge each chunk on the event loop before reading
another, so neither the reader buffer nor queued callbacks can grow
without bound.
- Cancel both bridge pumps on exit, allow up to one second for the
endpoint connection to flush, then abort the transport if it still holds
unsent bytes. A peer that stops reading cannot hold failure or
cancellation cleanup indefinitely.
- Resolve this installation's raven.exe, preserve task and session
isolation, hide endpoint credentials from grant representations, and
redact them in bridge diagnostics and session-start journal frames.
Remote dispatches do not receive the host's loopback token.
- Update capability reporting, CLI help, the remote ACP entry's
`machine` docstring, CONTEXT.md and the English/Chinese collaboration
guide. Add Windows CI coverage and exercise both Unix and TCP endpoints
on Linux.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Docker Linux, Debian Bookworm with procps and git, Python 3.12.12,
--init, a native Linux source copy, and a frozen environment: 783
related unit tests and all 3 real-server integration tests passed. The
source and test files were compared with the committed versions before
publication.
```sh
uv sync --frozen --extra dev
uv run --frozen --no-sync pytest tests/test_mcp_transport.py tests/test_mcp_bridge.py tests/test_mcp_endpoint.py tests/test_cli_mcp_commands.py tests/test_subagent_acp_mcp_lifecycle.py tests/test_node_client.py tests/test_acp_journal.py tests/test_subagent_acp_remote.py tests/test_subagent_manager.py tests/test_subagent_mcp_grant.py tests/test_subagent_third_party.py -q -n 4 --dist loadfile --tb=short --cov=raven --cov-report=json:/out/coverage.json --junitxml=/out/units.xml
uv run --frozen --no-sync pytest tests/integration/test_mcp_bridge_e2e.py -m integration -q -n 0 --tb=short --junitxml=/out/integration.xml
```
PR diff coverage is 98.26% (169/172 changed executable lines), above the
unchanged 90% threshold. On the host checkout, `$coverageReport` points
to the JSON exported by that Linux run:
```powershell
uv run --frozen --no-sync python scripts/coverage_gate.py --coverage-json $coverageReport diff --base-ref origin/main --head HEAD --threshold 90
```
Native Windows, Python 3.13.12, a separate frozen environment and
PYTHONUTF8=1: all 21 bridge unit tests passed; all 8 integration cases
passed, comprising 3 MCP bridge cases and 5 native command-line cases
retained in the Windows CI job.
```powershell
uv run --frozen --python 3.13 --no-sync pytest tests/test_mcp_bridge.py -q -n 0 --tb=short
uv run --frozen --python 3.13 --no-sync pytest tests/integration/test_mcp_bridge_e2e.py tests/integration/test_acp_mirasim_real_process.py -m integration -q -n 0 --tb=short
```
Seven regression cases cover paused input, resuming consumption, closing
a paused worker, a busy event loop, and teardown with pending TCP
writes: while the endpoint stays stalled, while it drains those writes,
and when a second cancel arrives after that drain finished. Aborting the
endpoint after a close that finished was wrong: the selector transport
has already run connection_lost and dropped its loop, and its
_force_close returns early only on _conn_lost, which a drained close
does not raise, so the bridge exited 1 at teardown on Linux and macOS.
The abort now depends on whether the endpoint's write buffer still holds
bytes, since a closing transport empties it before calling
connection_lost. A flag set once the close returned would miss the
second cancel, and that case fails with one. The stalled-endpoint
teardown case pauses the endpoint's reading before the frames are fed:
it used to wait for the socket to stop reading through its own flow
control, which took 1.87 s of the case's 2 s budget on Linux, so a busy
runner timed out before the teardown ran. It is now 0.17 s per case, and
20 consecutive runs under 32 busy loops on 16 cores all passed. The
original teardown and buffering cases fail without the fixes. At this
head, making the stdin worker read again without waiting for its
previous chunk to be delivered fails 3 cases, removing the close
deadline fails 2, and never aborting fails 2; aborting unconditionally
fails both drained cases. These mutations ran on a disposable copy of
the source tree, not on the committed files.
The unreachable-upstream integration test still uses its original
1.5-second deadline. Its Windows fixture bypasses the venv redirector
while preserving the venv, so an inherited stdout handle cannot mask
EOF.
Full-PR pre-commit, commitlint, commit-message, source-language, asset
and whitespace checks passed. The bridge type check passed and all 10
import contracts were kept:
```powershell
uv run --frozen --no-sync pre-commit run --from-ref origin/main --to-ref HEAD
npx --no-install commitlint --from origin/main --to HEAD --config commitlint.config.cjs
uv run --frozen --no-sync python scripts/check_commit_messages.py origin/main..HEAD
uv run --frozen --no-sync python scripts/check_large_files.py origin/main...HEAD
uv run --frozen --no-sync python scripts/check_source_language.py origin/main...HEAD
uv run --frozen --no-sync ty check raven/mcp/bridge.py
uv run --frozen --no-sync lint-imports
git diff --check origin/main...HEAD
```
These are focused local results. Remote dispatch tests use the
repository's SSH stand-in and real node processes, not an external
machine. POSIX-dependent paths are covered on Linux; no full-suite
Windows pass, live LLM run or sandbox VM run is claimed. Existing
Windows path, shell and journal portability outside this MCP change
remains out of scope.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
Windows dispatches can now use granted MCP servers. Each fallback
endpoint opens an ephemeral loopback listener with a bounded
authentication handshake and its own random token. Upstream definitions
and credentials remain on the host, and remote dispatches receive no
loopback token.
Input now pauses when a bridge cannot forward requests. During shutdown,
a connection that cannot flush within one second is aborted and its
remaining queued bytes can be discarded. This bounds cleanup when a peer
has stopped reading.
Unix sockets remain the preferred transport, and the Raven node wire
protocol is unchanged. No dependencies, configuration fields or
migrations are added. Reverting the change restores the previous Windows
MCP limitation.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Fixes #962
---------
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-19 19:58:55 +08:00
|
|
|
"""An authenticated loopback endpoint, with the frames and writers it accepted."""
|
|
|
|
|
from raven.mcp.transport import TOKEN_BYTES, authenticate
|
|
|
|
|
|
|
|
|
|
token = secrets.token_hex(TOKEN_BYTES)
|
feat(*): lend the host's MCP servers to an acp agent on a machine through its node (#947)
## Summary
Adds MCP lending for remote ACP agents on top of the machine and
recovery
services already merged in #920, #922 and #934. The branch includes
current
main and its default-on registered-machine behavior.
What this does: an acp entry on a registered machine is handed the MCP
servers its row grants, the way an entry on this computer is, and the
servers themselves stay here with their definitions and keys.
- The agent on the machine runs the node's own bridge as a stdio MCP
server (`python -I -m raven.node --mcp --dir <node dir> --task <task>
--lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte
pumps around a connection to the daemon's socket, no MCP parsed, no
credential held, the standard library only.
- The daemon carries the bridge's frames as a *stream* to the host
holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6,
`raven/node/protocol.py`, `raven/node/server.py`). The daemon tells
which
task a bridge is by the process that connected -- the peer pid off the
socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the
parent chain to the agent the daemon started, which the bridge is under
and no other task's process is; the kernel supplies that pid and a peer
cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the
task it names must be the one the connection is a process of. Two tasks
on
one machine run as one user, so no name or file a bridge could present
would tell them apart; the connecting process does. One stream per
bridge, at most 16 per task; a frame
travels in pieces of at most 256 KiB characters, the last marked
(`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the
daemon relays and keeps nothing. A stream ends with either end, the
host's link or the task, and the other end is told.
- The host opens the endpoints for a machine row exactly as for a row
here
(`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in
the host's sandbox, HTTP auth added here), binds them to the machine,
dispatch and acquired agent task
(`NodePool.lend`/`unlend`) before handing out bridge stanzas. On
`mcp/open`
the host checks the daemon-authenticated task against that binding
before
connecting; it rechecks the grant and link after the socket opens. An
unlend, replacement grant or replaced link cannot authorize that pending
connection. The host then pumps the frames both ways
(`raven/node/client.py`).
`node/hello` names the bridge; a node serving without a daemon
(`--stdio`)
names none, and the dispatch then goes without MCP and says so.
- The reply closes with the servers lent and the machine's node, or with
why none were. Remote lending requires Unix-socket support on the host:
on Windows it is withheld before endpoint setup, the task continues, and
the reply explains why. The roster's "injectable MCP" matches that
support
(`session_mcp_delivered`). The cli lane on a machine still
withholds its servers (the next change); the openai lane has no tools.
Not in this change: the cli lane's MCP file on the machine; a stream
that
survives the link going (a bridge sees its server end, the agent reports
the call failed, and the next dispatch works again); MCP for a run
picked
up after this computer restarted (#934): it reports its calls fail.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Follow-up fix 05eaba67a, after the c9de53947 review:
- The nine tests the Windows review named now pin the host fact through
one
fixture, `unix_socket_host` (`tests/conftest.py`), which sets
`raven.mcp.bridge.unix_sockets_available` to true and re-points every
`raven.*` module that imported it by name; their bodies are unchanged.
- Found complete, not just the nine: the whole suite was run with
`asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted
before collection (a pytest plugin, controller and xdist workers), at
the
merge base `6b938a426` and at `c9de53947`, and the two failure sets
compared. 17 node ids failed at the head and not at the base: the nine,
and eight this change added that run the transport for real (the bridge
module, and two lending tests through the stand-in machine). Those eight
are guarded by host capability instead of pinned: the bridge module
skips
when `sys.platform == "win32" or not unix_sockets_available()`, the two
lending tests carry the existing `skipif(not unix_sockets_available())`.
After the fix the same run fails the same 66 node ids at the head as at
the base: nothing this change adds or touches fails on a host without
unix sockets.
- A bridge whose daemon side goes ends as streams end. The host can
close a
stream while a frame of the agent's is on its way; the daemon writes
`mcp/close` and hangs up, and the frame meets the closed socket, or the
read side is reset first. That exited 1 with `BrokenPipeError` or
`ConnectionResetError`; a connection-level end is now exit 0 with
nothing
said, a frame past the cap still exits 1 with its reason, both pumps are
awaited before the socket closes, and a daemon gone before the stream
opens is one sentence, not a traceback. Four deterministic cases with a
stand-in connection fail on `c9de53947` and pass here.
- The bridge test's stand-in agent outlives its bridge, as an agent
outlives
one of its MCP servers, so the later parts of
`test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins`
no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80
here.
- The focused set (node, MCP, remote, manager and roster files) -- 935
passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
simulation cases, which fail the same way on main on this computer);
diff coverage against main 97.67% (420/430). Real machine, node
reinstalled with this tree in a directory of the check's own: a stdlib
ACP
agent's bridge reached a host-held server and 1 MiB and 8 MiB results
crossed whole, bridge exit 0; nothing was left on the machine.
Follow-up fix c9de53947, after the same-user token finding:
- The daemon no longer uses a token. `agent/launch` mints and writes
none,
`mcp.token` and the `hmac` check are gone, and `Daemon.open_stream`
identifies the bridge by the connecting process instead (`_peer_pid` +
`Daemon.task_of`). The same-user path the reviewer kept open -- a peer
reading another task's `mcp.token` and opening its stream -- is closed,
because identity is the connection the kernel reports, not a readable
secret.
- Tests, each failing on the unfixed tree first: a bridge placed under
task
B that names task A is refused (`the connection is a process of task
'agent@box-2', not 'agent@box-1'`), as are a process under no task, an
unplaced connection, and a task the daemon does not run
(`tests/test_node_server.py`); a real-OS test launches an agent, takes
its
real child, and asserts the child resolves to the task while the test's
own
process resolves to none (runs on Linux in CI); `_peer_pid` is checked
against a real socketpair; the in-process and subprocess bridge tests
now
identify by the connecting process, and a bridge this test spawns
itself,
or another task's agent spawns, never opens the stream.
- Windows withholding now covers a row on this computer too, not only a
machine row: `session_mcp_delivered` drops the `machine and` guard and
answers no whenever `unix_sockets_available()` is false, so a local acp
row is no longer advertised as injectable and no longer raises at
dispatch
on a host without the transport; `_mcp_endpoints` yields nothing before
opening an endpoint and `_mcp_note` explains it. The two tests the
Windows
review named now guard on `unix_sockets_available()`, and the
withholding
test is parameterized over a machine row and a local row.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over the node/MCP/remote suite -- 97.63% changed-line coverage
(412/422).
`ruff`, `ty` on the changed modules, and the import-cycle budget pass.
- Real machine (ravenx2, this head, node reinstalled, digest
fdbe132e76f0):
a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB
and
an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the
connecting-process identity admits the legitimate bridge end to end on
real Linux; the cross-task refusal on Linux is the real-OS test above,
which CI runs there.
Earlier review fixes (4dc5b3e1f, the task-to-grant binding). Its
host-side
binding of each dispatch's endpoints to the acquired agent task is kept;
the
daemon-side token this run verified was since replaced by the
connecting-process
identity above (c9de53947), so the token-named tests and reproduction
below no
longer describe the current tree.
- `uv run --no-sync pytest tests/test_node_client.py
tests/test_node_server.py tests/test_node_bridge.py
tests/test_node_relay.py tests/test_node_bundle.py
tests/test_node_install.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py
tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict
--cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report=
--tb=short` -- 1495 passed and one minimal-node stdio response test
failed;
strict idle accounting also flagged the assembly test. Both cases pass
together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or
timing limits were weakened. The new authorization, revocation and host
compatibility tests all pass.
- The five focused cases failed on the unfixed implementation first: a
different authenticated task selecting another grant, a stream with no
task, each missing Unix socket API, and an astral Unicode chunk over the
node's byte limit. Regressions also cover revocation, grant replacement
and link replacement while an endpoint connection is opening, and a real
remote turn continuing when both host Unix socket APIs are absent.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 97.94% (381/389 changed executable lines); this fix alone is 100%
(40/40). Python lint, type, dependency
and import checks pass; all 10 import contracts hold. The scope checks
for commit messages, source language and file sizes pass. Both RPC
generators pass `--check` after integrating current main.
- A real Linux node with two concurrent tasks and a host loopback
endpoint:
before, the attacker's own task token plus the victim's public grant
names returns the victim endpoint's marker; after, it returns no data
and the endpoint is never reached. Its own grant works, unknown and
revoked grants fail, and the same attack remains denied after the relay
is killed and an ordinary ping reattaches both tasks. A full chunk of
262144 astral characters was lost/corrupted before and arrives intact
after. The final documented source tree was reinstalled and checked
again.
- Real ACP dispatch through the node's bridge reaches a host-held stdio
MCP tool and returns its marker; its grant is removed when the turn
ends.
With the host Unix socket APIs removed to simulate Windows, the remote
agent still completes, receives no MCP, explains the host limitation,
and the host MCP ledger records no extra call. This is not a native
Windows run. Dedicated test tasks, host endpoints and bridge processes
are cleaned up.
- CONTEXT.md, module/field docstrings, both collaboration guides, the
external design notes and this description were checked against the
implementation. Main's no-debug-switch behavior and full-turn recovery
documentation are retained.
Full combined-tree verification: `uv run --no-sync pytest -n 4
--tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure
reproduces on current main in a separate checkout: host CLI color
rendering,
LibreOffice detection and the two simulation cases. No other test fails.
The minimal-node response test that failed in the focused parallel
selection
passes in this full run. Diff coverage: 97.94% (381/389 executable
changed lines).
Earlier author verification:
- `uv run pytest tests/test_node_bridge.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_bundle.py tests/test_node_install.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py`
-- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge
and a stdio node naming none, the token written owner-only beside the
record, a stream opened with the task's token and refused with another
or for a task not running, refused with no host holding the task,
refused past 16 per task, one per bridge connection, the pieces of a
frame crossing both ways as they are and a stranger carrying nothing,
and every way a stream ends (the bridge closing, the host closing, the
bridge's connection ending, the host's link ending, the agent ending)
with the other end told; the bridge as a process against a real daemon
over its socket: a frame to the host and one back whole, a large frame
each way in pieces, the agent closing its stdin ending the stream and
the bridge leaving clean, the host closing the stream ending the bridge,
a wrong token and a missing daemon each refused on stderr with exit 1;
the stand-in machine with the real daemon and the real bridge: an acp
stub there that connects to its `mcpServers` and calls a tool reaches a
stdio server here and answers with its result, the stanza handed to it
naming the node's bridge, the task, the dispatch and the server and
nothing of the server's definition or socket, the ledger here showing
one call, no bridge outliving the dispatch; the reply's note and the
roster predicate for a machine row.
- `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed,
4 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`,
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`
and
`test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`,
which fail
the same way on main on this machine.
- Real machine (a registered Linux machine with Claude Code signed in,
the node reinstalled with the code before and after):
A28, before (the
code of #934's head and its node): the turn closed with "MCP servers
'probe' were not lent", Claude Code said no tool of that name was
connected, and the server here saw no call; after: `PROBE_OK:r2-lend`
in 9.4 s, the server here recorded the call, and the turn closed with
"lent to acp agent ... through the Raven node on machine ...". A34:
after
the turn the machine's process list shows no bridge, a grep of its node
directory finds nothing of the server's command or of this computer's
socket directory, and no endpoint socket is left here. A29: with the
tool switched off here, Claude Code answered UNAVAILABLE and the server
saw no call. A30: a 40 s tool call under way, the bridge visible in the
machine's process list, then cancelled: 3 s later no bridge there and
no socket here. A31: the relay ssh killed during a 20 s call: the turn
ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next
dispatch reached the server again in 7.6 s. A32: five calls in one turn
all reached the server and came back; an 8 MiB answer: the server here
produced it, and a plain ACP agent on the machine (a stdlib stub that
speaks the stanza's bridge directly) received it whole through the node
-- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude
Code itself closed the server on that answer and reported the call
failed: the agent's own cap on a tool result, not the link's.
- `ruff`, the commit-message and PR-body checks -- clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese collaboration guides describe lending,
task-bound authorization,
Windows withholding, and the existing CLI/restart limitations.
## Risk
Registered-machine support is enabled by configuration, without the
removed
trial environment switch. An ACP entry on a machine receives its granted
MCP servers on a Unix-socket-capable host; the host's normal MCP policy
still
applies. This intentionally exposes those tools to the selected remote
agent.
The server definition, credentials and endpoint paths remain on the
host.
The daemon identifies a bridge's task by the connecting process (the
peer
pid the kernel reports, walked to the agent it started), not by any
token or
name the bridge presents, so a peer task sharing the OS account cannot
open
another task's stream; the host then authorizes that task against the
dispatch/server grant. A system that reports no peer pid opens no
stream.
On Windows hosts remote lending is withheld explicitly, without aborting
the
remote task. Protocol 6 remains the MCP wire version; host and node must
also
have the same code digest, so updated nodes must be reinstalled.
Rollback is
a revert; the machine then carries no stream and receives no bridge
stanza.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#934 (merged); #920 and #922 (merged).
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-15 15:37:50 +08:00
|
|
|
got: list[bytes] = []
|
|
|
|
|
writers: list[asyncio.StreamWriter] = []
|
|
|
|
|
|
|
|
|
|
async def serve(reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
|
feat: enable MCP lending on Windows hosts (#972)
## Summary
Windows hosts can lend granted MCP servers to local and machine-based
ACP agents through authenticated loopback endpoints. Hosts with Unix
sockets retain that transport. Server definitions, upstream credentials,
sandbox policy and disabled-tool enforcement stay on the host.
- Share one connector between the local bridge and node client. Bind TCP
endpoints only to 127.0.0.1 and authenticate with a fresh per-endpoint
token before opening an upstream.
- Read inherited Windows stdin pipes through a daemon worker with
backpressure. Acknowledge each chunk on the event loop before reading
another, so neither the reader buffer nor queued callbacks can grow
without bound.
- Cancel both bridge pumps on exit, allow up to one second for the
endpoint connection to flush, then abort the transport if it still holds
unsent bytes. A peer that stops reading cannot hold failure or
cancellation cleanup indefinitely.
- Resolve this installation's raven.exe, preserve task and session
isolation, hide endpoint credentials from grant representations, and
redact them in bridge diagnostics and session-start journal frames.
Remote dispatches do not receive the host's loopback token.
- Update capability reporting, CLI help, the remote ACP entry's
`machine` docstring, CONTEXT.md and the English/Chinese collaboration
guide. Add Windows CI coverage and exercise both Unix and TCP endpoints
on Linux.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Docker Linux, Debian Bookworm with procps and git, Python 3.12.12,
--init, a native Linux source copy, and a frozen environment: 783
related unit tests and all 3 real-server integration tests passed. The
source and test files were compared with the committed versions before
publication.
```sh
uv sync --frozen --extra dev
uv run --frozen --no-sync pytest tests/test_mcp_transport.py tests/test_mcp_bridge.py tests/test_mcp_endpoint.py tests/test_cli_mcp_commands.py tests/test_subagent_acp_mcp_lifecycle.py tests/test_node_client.py tests/test_acp_journal.py tests/test_subagent_acp_remote.py tests/test_subagent_manager.py tests/test_subagent_mcp_grant.py tests/test_subagent_third_party.py -q -n 4 --dist loadfile --tb=short --cov=raven --cov-report=json:/out/coverage.json --junitxml=/out/units.xml
uv run --frozen --no-sync pytest tests/integration/test_mcp_bridge_e2e.py -m integration -q -n 0 --tb=short --junitxml=/out/integration.xml
```
PR diff coverage is 98.26% (169/172 changed executable lines), above the
unchanged 90% threshold. On the host checkout, `$coverageReport` points
to the JSON exported by that Linux run:
```powershell
uv run --frozen --no-sync python scripts/coverage_gate.py --coverage-json $coverageReport diff --base-ref origin/main --head HEAD --threshold 90
```
Native Windows, Python 3.13.12, a separate frozen environment and
PYTHONUTF8=1: all 21 bridge unit tests passed; all 8 integration cases
passed, comprising 3 MCP bridge cases and 5 native command-line cases
retained in the Windows CI job.
```powershell
uv run --frozen --python 3.13 --no-sync pytest tests/test_mcp_bridge.py -q -n 0 --tb=short
uv run --frozen --python 3.13 --no-sync pytest tests/integration/test_mcp_bridge_e2e.py tests/integration/test_acp_mirasim_real_process.py -m integration -q -n 0 --tb=short
```
Seven regression cases cover paused input, resuming consumption, closing
a paused worker, a busy event loop, and teardown with pending TCP
writes: while the endpoint stays stalled, while it drains those writes,
and when a second cancel arrives after that drain finished. Aborting the
endpoint after a close that finished was wrong: the selector transport
has already run connection_lost and dropped its loop, and its
_force_close returns early only on _conn_lost, which a drained close
does not raise, so the bridge exited 1 at teardown on Linux and macOS.
The abort now depends on whether the endpoint's write buffer still holds
bytes, since a closing transport empties it before calling
connection_lost. A flag set once the close returned would miss the
second cancel, and that case fails with one. The stalled-endpoint
teardown case pauses the endpoint's reading before the frames are fed:
it used to wait for the socket to stop reading through its own flow
control, which took 1.87 s of the case's 2 s budget on Linux, so a busy
runner timed out before the teardown ran. It is now 0.17 s per case, and
20 consecutive runs under 32 busy loops on 16 cores all passed. The
original teardown and buffering cases fail without the fixes. At this
head, making the stdin worker read again without waiting for its
previous chunk to be delivered fails 3 cases, removing the close
deadline fails 2, and never aborting fails 2; aborting unconditionally
fails both drained cases. These mutations ran on a disposable copy of
the source tree, not on the committed files.
The unreachable-upstream integration test still uses its original
1.5-second deadline. Its Windows fixture bypasses the venv redirector
while preserving the venv, so an inherited stdout handle cannot mask
EOF.
Full-PR pre-commit, commitlint, commit-message, source-language, asset
and whitespace checks passed. The bridge type check passed and all 10
import contracts were kept:
```powershell
uv run --frozen --no-sync pre-commit run --from-ref origin/main --to-ref HEAD
npx --no-install commitlint --from origin/main --to HEAD --config commitlint.config.cjs
uv run --frozen --no-sync python scripts/check_commit_messages.py origin/main..HEAD
uv run --frozen --no-sync python scripts/check_large_files.py origin/main...HEAD
uv run --frozen --no-sync python scripts/check_source_language.py origin/main...HEAD
uv run --frozen --no-sync ty check raven/mcp/bridge.py
uv run --frozen --no-sync lint-imports
git diff --check origin/main...HEAD
```
These are focused local results. Remote dispatch tests use the
repository's SSH stand-in and real node processes, not an external
machine. POSIX-dependent paths are covered on Linux; no full-suite
Windows pass, live LLM run or sandbox VM run is claimed. Existing
Windows path, shell and journal portability outside this MCP change
remains out of scope.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
Windows dispatches can now use granted MCP servers. Each fallback
endpoint opens an ephemeral loopback listener with a bounded
authentication handshake and its own random token. Upstream definitions
and credentials remain on the host, and remote dispatches receive no
loopback token.
Input now pauses when a bridge cannot forward requests. During shutdown,
a connection that cannot flush within one second is aborted and its
remaining queued bytes can be discarded. This bounds cleanup when a peer
has stopped reading.
Unix sockets remain the preferred transport, and the Raven node wire
protocol is unchanged. No dependencies, configuration fields or
migrations are added. Reverting the change restores the previous Windows
MCP limitation.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Fixes #962
---------
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-19 19:58:55 +08:00
|
|
|
try:
|
|
|
|
|
if not await authenticate(reader, writer, token):
|
|
|
|
|
return
|
|
|
|
|
writers.append(writer)
|
|
|
|
|
while line := await reader.readline():
|
|
|
|
|
got.append(line)
|
|
|
|
|
finally:
|
|
|
|
|
writer.close()
|
|
|
|
|
|
|
|
|
|
listening = await asyncio.start_server(serve, "127.0.0.1", 0)
|
|
|
|
|
port = listening.sockets[0].getsockname()[1]
|
|
|
|
|
address = f"tcp://127.0.0.1:{port}/{token}"
|
feat(*): lend the host's MCP servers to an acp agent on a machine through its node (#947)
## Summary
Adds MCP lending for remote ACP agents on top of the machine and
recovery
services already merged in #920, #922 and #934. The branch includes
current
main and its default-on registered-machine behavior.
What this does: an acp entry on a registered machine is handed the MCP
servers its row grants, the way an entry on this computer is, and the
servers themselves stay here with their definitions and keys.
- The agent on the machine runs the node's own bridge as a stdio MCP
server (`python -I -m raven.node --mcp --dir <node dir> --task <task>
--lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte
pumps around a connection to the daemon's socket, no MCP parsed, no
credential held, the standard library only.
- The daemon carries the bridge's frames as a *stream* to the host
holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6,
`raven/node/protocol.py`, `raven/node/server.py`). The daemon tells
which
task a bridge is by the process that connected -- the peer pid off the
socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the
parent chain to the agent the daemon started, which the bridge is under
and no other task's process is; the kernel supplies that pid and a peer
cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the
task it names must be the one the connection is a process of. Two tasks
on
one machine run as one user, so no name or file a bridge could present
would tell them apart; the connecting process does. One stream per
bridge, at most 16 per task; a frame
travels in pieces of at most 256 KiB characters, the last marked
(`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the
daemon relays and keeps nothing. A stream ends with either end, the
host's link or the task, and the other end is told.
- The host opens the endpoints for a machine row exactly as for a row
here
(`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in
the host's sandbox, HTTP auth added here), binds them to the machine,
dispatch and acquired agent task
(`NodePool.lend`/`unlend`) before handing out bridge stanzas. On
`mcp/open`
the host checks the daemon-authenticated task against that binding
before
connecting; it rechecks the grant and link after the socket opens. An
unlend, replacement grant or replaced link cannot authorize that pending
connection. The host then pumps the frames both ways
(`raven/node/client.py`).
`node/hello` names the bridge; a node serving without a daemon
(`--stdio`)
names none, and the dispatch then goes without MCP and says so.
- The reply closes with the servers lent and the machine's node, or with
why none were. Remote lending requires Unix-socket support on the host:
on Windows it is withheld before endpoint setup, the task continues, and
the reply explains why. The roster's "injectable MCP" matches that
support
(`session_mcp_delivered`). The cli lane on a machine still
withholds its servers (the next change); the openai lane has no tools.
Not in this change: the cli lane's MCP file on the machine; a stream
that
survives the link going (a bridge sees its server end, the agent reports
the call failed, and the next dispatch works again); MCP for a run
picked
up after this computer restarted (#934): it reports its calls fail.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Follow-up fix 05eaba67a, after the c9de53947 review:
- The nine tests the Windows review named now pin the host fact through
one
fixture, `unix_socket_host` (`tests/conftest.py`), which sets
`raven.mcp.bridge.unix_sockets_available` to true and re-points every
`raven.*` module that imported it by name; their bodies are unchanged.
- Found complete, not just the nine: the whole suite was run with
`asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted
before collection (a pytest plugin, controller and xdist workers), at
the
merge base `6b938a426` and at `c9de53947`, and the two failure sets
compared. 17 node ids failed at the head and not at the base: the nine,
and eight this change added that run the transport for real (the bridge
module, and two lending tests through the stand-in machine). Those eight
are guarded by host capability instead of pinned: the bridge module
skips
when `sys.platform == "win32" or not unix_sockets_available()`, the two
lending tests carry the existing `skipif(not unix_sockets_available())`.
After the fix the same run fails the same 66 node ids at the head as at
the base: nothing this change adds or touches fails on a host without
unix sockets.
- A bridge whose daemon side goes ends as streams end. The host can
close a
stream while a frame of the agent's is on its way; the daemon writes
`mcp/close` and hangs up, and the frame meets the closed socket, or the
read side is reset first. That exited 1 with `BrokenPipeError` or
`ConnectionResetError`; a connection-level end is now exit 0 with
nothing
said, a frame past the cap still exits 1 with its reason, both pumps are
awaited before the socket closes, and a daemon gone before the stream
opens is one sentence, not a traceback. Four deterministic cases with a
stand-in connection fail on `c9de53947` and pass here.
- The bridge test's stand-in agent outlives its bridge, as an agent
outlives
one of its MCP servers, so the later parts of
`test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins`
no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80
here.
- The focused set (node, MCP, remote, manager and roster files) -- 935
passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
simulation cases, which fail the same way on main on this computer);
diff coverage against main 97.67% (420/430). Real machine, node
reinstalled with this tree in a directory of the check's own: a stdlib
ACP
agent's bridge reached a host-held server and 1 MiB and 8 MiB results
crossed whole, bridge exit 0; nothing was left on the machine.
Follow-up fix c9de53947, after the same-user token finding:
- The daemon no longer uses a token. `agent/launch` mints and writes
none,
`mcp.token` and the `hmac` check are gone, and `Daemon.open_stream`
identifies the bridge by the connecting process instead (`_peer_pid` +
`Daemon.task_of`). The same-user path the reviewer kept open -- a peer
reading another task's `mcp.token` and opening its stream -- is closed,
because identity is the connection the kernel reports, not a readable
secret.
- Tests, each failing on the unfixed tree first: a bridge placed under
task
B that names task A is refused (`the connection is a process of task
'agent@box-2', not 'agent@box-1'`), as are a process under no task, an
unplaced connection, and a task the daemon does not run
(`tests/test_node_server.py`); a real-OS test launches an agent, takes
its
real child, and asserts the child resolves to the task while the test's
own
process resolves to none (runs on Linux in CI); `_peer_pid` is checked
against a real socketpair; the in-process and subprocess bridge tests
now
identify by the connecting process, and a bridge this test spawns
itself,
or another task's agent spawns, never opens the stream.
- Windows withholding now covers a row on this computer too, not only a
machine row: `session_mcp_delivered` drops the `machine and` guard and
answers no whenever `unix_sockets_available()` is false, so a local acp
row is no longer advertised as injectable and no longer raises at
dispatch
on a host without the transport; `_mcp_endpoints` yields nothing before
opening an endpoint and `_mcp_note` explains it. The two tests the
Windows
review named now guard on `unix_sockets_available()`, and the
withholding
test is parameterized over a machine row and a local row.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over the node/MCP/remote suite -- 97.63% changed-line coverage
(412/422).
`ruff`, `ty` on the changed modules, and the import-cycle budget pass.
- Real machine (ravenx2, this head, node reinstalled, digest
fdbe132e76f0):
a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB
and
an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the
connecting-process identity admits the legitimate bridge end to end on
real Linux; the cross-task refusal on Linux is the real-OS test above,
which CI runs there.
Earlier review fixes (4dc5b3e1f, the task-to-grant binding). Its
host-side
binding of each dispatch's endpoints to the acquired agent task is kept;
the
daemon-side token this run verified was since replaced by the
connecting-process
identity above (c9de53947), so the token-named tests and reproduction
below no
longer describe the current tree.
- `uv run --no-sync pytest tests/test_node_client.py
tests/test_node_server.py tests/test_node_bridge.py
tests/test_node_relay.py tests/test_node_bundle.py
tests/test_node_install.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py
tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict
--cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report=
--tb=short` -- 1495 passed and one minimal-node stdio response test
failed;
strict idle accounting also flagged the assembly test. Both cases pass
together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or
timing limits were weakened. The new authorization, revocation and host
compatibility tests all pass.
- The five focused cases failed on the unfixed implementation first: a
different authenticated task selecting another grant, a stream with no
task, each missing Unix socket API, and an astral Unicode chunk over the
node's byte limit. Regressions also cover revocation, grant replacement
and link replacement while an endpoint connection is opening, and a real
remote turn continuing when both host Unix socket APIs are absent.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 97.94% (381/389 changed executable lines); this fix alone is 100%
(40/40). Python lint, type, dependency
and import checks pass; all 10 import contracts hold. The scope checks
for commit messages, source language and file sizes pass. Both RPC
generators pass `--check` after integrating current main.
- A real Linux node with two concurrent tasks and a host loopback
endpoint:
before, the attacker's own task token plus the victim's public grant
names returns the victim endpoint's marker; after, it returns no data
and the endpoint is never reached. Its own grant works, unknown and
revoked grants fail, and the same attack remains denied after the relay
is killed and an ordinary ping reattaches both tasks. A full chunk of
262144 astral characters was lost/corrupted before and arrives intact
after. The final documented source tree was reinstalled and checked
again.
- Real ACP dispatch through the node's bridge reaches a host-held stdio
MCP tool and returns its marker; its grant is removed when the turn
ends.
With the host Unix socket APIs removed to simulate Windows, the remote
agent still completes, receives no MCP, explains the host limitation,
and the host MCP ledger records no extra call. This is not a native
Windows run. Dedicated test tasks, host endpoints and bridge processes
are cleaned up.
- CONTEXT.md, module/field docstrings, both collaboration guides, the
external design notes and this description were checked against the
implementation. Main's no-debug-switch behavior and full-turn recovery
documentation are retained.
Full combined-tree verification: `uv run --no-sync pytest -n 4
--tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure
reproduces on current main in a separate checkout: host CLI color
rendering,
LibreOffice detection and the two simulation cases. No other test fails.
The minimal-node response test that failed in the focused parallel
selection
passes in this full run. Diff coverage: 97.94% (381/389 executable
changed lines).
Earlier author verification:
- `uv run pytest tests/test_node_bridge.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_bundle.py tests/test_node_install.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py`
-- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge
and a stdio node naming none, the token written owner-only beside the
record, a stream opened with the task's token and refused with another
or for a task not running, refused with no host holding the task,
refused past 16 per task, one per bridge connection, the pieces of a
frame crossing both ways as they are and a stranger carrying nothing,
and every way a stream ends (the bridge closing, the host closing, the
bridge's connection ending, the host's link ending, the agent ending)
with the other end told; the bridge as a process against a real daemon
over its socket: a frame to the host and one back whole, a large frame
each way in pieces, the agent closing its stdin ending the stream and
the bridge leaving clean, the host closing the stream ending the bridge,
a wrong token and a missing daemon each refused on stderr with exit 1;
the stand-in machine with the real daemon and the real bridge: an acp
stub there that connects to its `mcpServers` and calls a tool reaches a
stdio server here and answers with its result, the stanza handed to it
naming the node's bridge, the task, the dispatch and the server and
nothing of the server's definition or socket, the ledger here showing
one call, no bridge outliving the dispatch; the reply's note and the
roster predicate for a machine row.
- `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed,
4 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`,
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`
and
`test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`,
which fail
the same way on main on this machine.
- Real machine (a registered Linux machine with Claude Code signed in,
the node reinstalled with the code before and after):
A28, before (the
code of #934's head and its node): the turn closed with "MCP servers
'probe' were not lent", Claude Code said no tool of that name was
connected, and the server here saw no call; after: `PROBE_OK:r2-lend`
in 9.4 s, the server here recorded the call, and the turn closed with
"lent to acp agent ... through the Raven node on machine ...". A34:
after
the turn the machine's process list shows no bridge, a grep of its node
directory finds nothing of the server's command or of this computer's
socket directory, and no endpoint socket is left here. A29: with the
tool switched off here, Claude Code answered UNAVAILABLE and the server
saw no call. A30: a 40 s tool call under way, the bridge visible in the
machine's process list, then cancelled: 3 s later no bridge there and
no socket here. A31: the relay ssh killed during a 20 s call: the turn
ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next
dispatch reached the server again in 7.6 s. A32: five calls in one turn
all reached the server and came back; an 8 MiB answer: the server here
produced it, and a plain ACP agent on the machine (a stdlib stub that
speaks the stanza's bridge directly) received it whole through the node
-- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude
Code itself closed the server on that answer and reported the call
failed: the agent's own cap on a tool result, not the link's.
- `ruff`, the commit-message and PR-body checks -- clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese collaboration guides describe lending,
task-bound authorization,
Windows withholding, and the existing CLI/restart limitations.
## Risk
Registered-machine support is enabled by configuration, without the
removed
trial environment switch. An ACP entry on a machine receives its granted
MCP servers on a Unix-socket-capable host; the host's normal MCP policy
still
applies. This intentionally exposes those tools to the selected remote
agent.
The server definition, credentials and endpoint paths remain on the
host.
The daemon identifies a bridge's task by the connecting process (the
peer
pid the kernel reports, walked to the agent it started), not by any
token or
name the bridge presents, so a peer task sharing the OS account cannot
open
another task's stream; the host then authorizes that task against the
dispatch/server grant. A system that reports no peer pid opens no
stream.
On Windows hosts remote lending is withheld explicitly, without aborting
the
remote task. Protocol 6 remains the MCP wire version; host and node must
also
have the same code digest, so updated nodes must be reinstalled.
Rollback is
a revert; the machine then carries no stream and receives no bridge
stanza.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#934 (merged); #920 and #922 (merged).
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-15 15:37:50 +08:00
|
|
|
|
|
|
|
|
def close() -> None:
|
|
|
|
|
listening.close()
|
feat: enable MCP lending on Windows hosts (#972)
## Summary
Windows hosts can lend granted MCP servers to local and machine-based
ACP agents through authenticated loopback endpoints. Hosts with Unix
sockets retain that transport. Server definitions, upstream credentials,
sandbox policy and disabled-tool enforcement stay on the host.
- Share one connector between the local bridge and node client. Bind TCP
endpoints only to 127.0.0.1 and authenticate with a fresh per-endpoint
token before opening an upstream.
- Read inherited Windows stdin pipes through a daemon worker with
backpressure. Acknowledge each chunk on the event loop before reading
another, so neither the reader buffer nor queued callbacks can grow
without bound.
- Cancel both bridge pumps on exit, allow up to one second for the
endpoint connection to flush, then abort the transport if it still holds
unsent bytes. A peer that stops reading cannot hold failure or
cancellation cleanup indefinitely.
- Resolve this installation's raven.exe, preserve task and session
isolation, hide endpoint credentials from grant representations, and
redact them in bridge diagnostics and session-start journal frames.
Remote dispatches do not receive the host's loopback token.
- Update capability reporting, CLI help, the remote ACP entry's
`machine` docstring, CONTEXT.md and the English/Chinese collaboration
guide. Add Windows CI coverage and exercise both Unix and TCP endpoints
on Linux.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Docker Linux, Debian Bookworm with procps and git, Python 3.12.12,
--init, a native Linux source copy, and a frozen environment: 783
related unit tests and all 3 real-server integration tests passed. The
source and test files were compared with the committed versions before
publication.
```sh
uv sync --frozen --extra dev
uv run --frozen --no-sync pytest tests/test_mcp_transport.py tests/test_mcp_bridge.py tests/test_mcp_endpoint.py tests/test_cli_mcp_commands.py tests/test_subagent_acp_mcp_lifecycle.py tests/test_node_client.py tests/test_acp_journal.py tests/test_subagent_acp_remote.py tests/test_subagent_manager.py tests/test_subagent_mcp_grant.py tests/test_subagent_third_party.py -q -n 4 --dist loadfile --tb=short --cov=raven --cov-report=json:/out/coverage.json --junitxml=/out/units.xml
uv run --frozen --no-sync pytest tests/integration/test_mcp_bridge_e2e.py -m integration -q -n 0 --tb=short --junitxml=/out/integration.xml
```
PR diff coverage is 98.26% (169/172 changed executable lines), above the
unchanged 90% threshold. On the host checkout, `$coverageReport` points
to the JSON exported by that Linux run:
```powershell
uv run --frozen --no-sync python scripts/coverage_gate.py --coverage-json $coverageReport diff --base-ref origin/main --head HEAD --threshold 90
```
Native Windows, Python 3.13.12, a separate frozen environment and
PYTHONUTF8=1: all 21 bridge unit tests passed; all 8 integration cases
passed, comprising 3 MCP bridge cases and 5 native command-line cases
retained in the Windows CI job.
```powershell
uv run --frozen --python 3.13 --no-sync pytest tests/test_mcp_bridge.py -q -n 0 --tb=short
uv run --frozen --python 3.13 --no-sync pytest tests/integration/test_mcp_bridge_e2e.py tests/integration/test_acp_mirasim_real_process.py -m integration -q -n 0 --tb=short
```
Seven regression cases cover paused input, resuming consumption, closing
a paused worker, a busy event loop, and teardown with pending TCP
writes: while the endpoint stays stalled, while it drains those writes,
and when a second cancel arrives after that drain finished. Aborting the
endpoint after a close that finished was wrong: the selector transport
has already run connection_lost and dropped its loop, and its
_force_close returns early only on _conn_lost, which a drained close
does not raise, so the bridge exited 1 at teardown on Linux and macOS.
The abort now depends on whether the endpoint's write buffer still holds
bytes, since a closing transport empties it before calling
connection_lost. A flag set once the close returned would miss the
second cancel, and that case fails with one. The stalled-endpoint
teardown case pauses the endpoint's reading before the frames are fed:
it used to wait for the socket to stop reading through its own flow
control, which took 1.87 s of the case's 2 s budget on Linux, so a busy
runner timed out before the teardown ran. It is now 0.17 s per case, and
20 consecutive runs under 32 busy loops on 16 cores all passed. The
original teardown and buffering cases fail without the fixes. At this
head, making the stdin worker read again without waiting for its
previous chunk to be delivered fails 3 cases, removing the close
deadline fails 2, and never aborting fails 2; aborting unconditionally
fails both drained cases. These mutations ran on a disposable copy of
the source tree, not on the committed files.
The unreachable-upstream integration test still uses its original
1.5-second deadline. Its Windows fixture bypasses the venv redirector
while preserving the venv, so an inherited stdout handle cannot mask
EOF.
Full-PR pre-commit, commitlint, commit-message, source-language, asset
and whitespace checks passed. The bridge type check passed and all 10
import contracts were kept:
```powershell
uv run --frozen --no-sync pre-commit run --from-ref origin/main --to-ref HEAD
npx --no-install commitlint --from origin/main --to HEAD --config commitlint.config.cjs
uv run --frozen --no-sync python scripts/check_commit_messages.py origin/main..HEAD
uv run --frozen --no-sync python scripts/check_large_files.py origin/main...HEAD
uv run --frozen --no-sync python scripts/check_source_language.py origin/main...HEAD
uv run --frozen --no-sync ty check raven/mcp/bridge.py
uv run --frozen --no-sync lint-imports
git diff --check origin/main...HEAD
```
These are focused local results. Remote dispatch tests use the
repository's SSH stand-in and real node processes, not an external
machine. POSIX-dependent paths are covered on Linux; no full-suite
Windows pass, live LLM run or sandbox VM run is claimed. Existing
Windows path, shell and journal portability outside this MCP change
remains out of scope.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
Windows dispatches can now use granted MCP servers. Each fallback
endpoint opens an ephemeral loopback listener with a bounded
authentication handshake and its own random token. Upstream definitions
and credentials remain on the host, and remote dispatches receive no
loopback token.
Input now pauses when a bridge cannot forward requests. During shutdown,
a connection that cannot flush within one second is aborted and its
remaining queued bytes can be discarded. This bounds cleanup when a peer
has stopped reading.
Unix sockets remain the preferred transport, and the Raven node wire
protocol is unchanged. No dependencies, configuration fields or
migrations are added. Reverting the change restores the previous Windows
MCP limitation.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Fixes #962
---------
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-19 19:58:55 +08:00
|
|
|
for writer in writers:
|
|
|
|
|
writer.close()
|
feat(*): lend the host's MCP servers to an acp agent on a machine through its node (#947)
## Summary
Adds MCP lending for remote ACP agents on top of the machine and
recovery
services already merged in #920, #922 and #934. The branch includes
current
main and its default-on registered-machine behavior.
What this does: an acp entry on a registered machine is handed the MCP
servers its row grants, the way an entry on this computer is, and the
servers themselves stay here with their definitions and keys.
- The agent on the machine runs the node's own bridge as a stdio MCP
server (`python -I -m raven.node --mcp --dir <node dir> --task <task>
--lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte
pumps around a connection to the daemon's socket, no MCP parsed, no
credential held, the standard library only.
- The daemon carries the bridge's frames as a *stream* to the host
holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6,
`raven/node/protocol.py`, `raven/node/server.py`). The daemon tells
which
task a bridge is by the process that connected -- the peer pid off the
socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the
parent chain to the agent the daemon started, which the bridge is under
and no other task's process is; the kernel supplies that pid and a peer
cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the
task it names must be the one the connection is a process of. Two tasks
on
one machine run as one user, so no name or file a bridge could present
would tell them apart; the connecting process does. One stream per
bridge, at most 16 per task; a frame
travels in pieces of at most 256 KiB characters, the last marked
(`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the
daemon relays and keeps nothing. A stream ends with either end, the
host's link or the task, and the other end is told.
- The host opens the endpoints for a machine row exactly as for a row
here
(`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in
the host's sandbox, HTTP auth added here), binds them to the machine,
dispatch and acquired agent task
(`NodePool.lend`/`unlend`) before handing out bridge stanzas. On
`mcp/open`
the host checks the daemon-authenticated task against that binding
before
connecting; it rechecks the grant and link after the socket opens. An
unlend, replacement grant or replaced link cannot authorize that pending
connection. The host then pumps the frames both ways
(`raven/node/client.py`).
`node/hello` names the bridge; a node serving without a daemon
(`--stdio`)
names none, and the dispatch then goes without MCP and says so.
- The reply closes with the servers lent and the machine's node, or with
why none were. Remote lending requires Unix-socket support on the host:
on Windows it is withheld before endpoint setup, the task continues, and
the reply explains why. The roster's "injectable MCP" matches that
support
(`session_mcp_delivered`). The cli lane on a machine still
withholds its servers (the next change); the openai lane has no tools.
Not in this change: the cli lane's MCP file on the machine; a stream
that
survives the link going (a bridge sees its server end, the agent reports
the call failed, and the next dispatch works again); MCP for a run
picked
up after this computer restarted (#934): it reports its calls fail.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Follow-up fix 05eaba67a, after the c9de53947 review:
- The nine tests the Windows review named now pin the host fact through
one
fixture, `unix_socket_host` (`tests/conftest.py`), which sets
`raven.mcp.bridge.unix_sockets_available` to true and re-points every
`raven.*` module that imported it by name; their bodies are unchanged.
- Found complete, not just the nine: the whole suite was run with
`asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted
before collection (a pytest plugin, controller and xdist workers), at
the
merge base `6b938a426` and at `c9de53947`, and the two failure sets
compared. 17 node ids failed at the head and not at the base: the nine,
and eight this change added that run the transport for real (the bridge
module, and two lending tests through the stand-in machine). Those eight
are guarded by host capability instead of pinned: the bridge module
skips
when `sys.platform == "win32" or not unix_sockets_available()`, the two
lending tests carry the existing `skipif(not unix_sockets_available())`.
After the fix the same run fails the same 66 node ids at the head as at
the base: nothing this change adds or touches fails on a host without
unix sockets.
- A bridge whose daemon side goes ends as streams end. The host can
close a
stream while a frame of the agent's is on its way; the daemon writes
`mcp/close` and hangs up, and the frame meets the closed socket, or the
read side is reset first. That exited 1 with `BrokenPipeError` or
`ConnectionResetError`; a connection-level end is now exit 0 with
nothing
said, a frame past the cap still exits 1 with its reason, both pumps are
awaited before the socket closes, and a daemon gone before the stream
opens is one sentence, not a traceback. Four deterministic cases with a
stand-in connection fail on `c9de53947` and pass here.
- The bridge test's stand-in agent outlives its bridge, as an agent
outlives
one of its MCP servers, so the later parts of
`test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins`
no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80
here.
- The focused set (node, MCP, remote, manager and roster files) -- 935
passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
simulation cases, which fail the same way on main on this computer);
diff coverage against main 97.67% (420/430). Real machine, node
reinstalled with this tree in a directory of the check's own: a stdlib
ACP
agent's bridge reached a host-held server and 1 MiB and 8 MiB results
crossed whole, bridge exit 0; nothing was left on the machine.
Follow-up fix c9de53947, after the same-user token finding:
- The daemon no longer uses a token. `agent/launch` mints and writes
none,
`mcp.token` and the `hmac` check are gone, and `Daemon.open_stream`
identifies the bridge by the connecting process instead (`_peer_pid` +
`Daemon.task_of`). The same-user path the reviewer kept open -- a peer
reading another task's `mcp.token` and opening its stream -- is closed,
because identity is the connection the kernel reports, not a readable
secret.
- Tests, each failing on the unfixed tree first: a bridge placed under
task
B that names task A is refused (`the connection is a process of task
'agent@box-2', not 'agent@box-1'`), as are a process under no task, an
unplaced connection, and a task the daemon does not run
(`tests/test_node_server.py`); a real-OS test launches an agent, takes
its
real child, and asserts the child resolves to the task while the test's
own
process resolves to none (runs on Linux in CI); `_peer_pid` is checked
against a real socketpair; the in-process and subprocess bridge tests
now
identify by the connecting process, and a bridge this test spawns
itself,
or another task's agent spawns, never opens the stream.
- Windows withholding now covers a row on this computer too, not only a
machine row: `session_mcp_delivered` drops the `machine and` guard and
answers no whenever `unix_sockets_available()` is false, so a local acp
row is no longer advertised as injectable and no longer raises at
dispatch
on a host without the transport; `_mcp_endpoints` yields nothing before
opening an endpoint and `_mcp_note` explains it. The two tests the
Windows
review named now guard on `unix_sockets_available()`, and the
withholding
test is parameterized over a machine row and a local row.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over the node/MCP/remote suite -- 97.63% changed-line coverage
(412/422).
`ruff`, `ty` on the changed modules, and the import-cycle budget pass.
- Real machine (ravenx2, this head, node reinstalled, digest
fdbe132e76f0):
a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB
and
an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the
connecting-process identity admits the legitimate bridge end to end on
real Linux; the cross-task refusal on Linux is the real-OS test above,
which CI runs there.
Earlier review fixes (4dc5b3e1f, the task-to-grant binding). Its
host-side
binding of each dispatch's endpoints to the acquired agent task is kept;
the
daemon-side token this run verified was since replaced by the
connecting-process
identity above (c9de53947), so the token-named tests and reproduction
below no
longer describe the current tree.
- `uv run --no-sync pytest tests/test_node_client.py
tests/test_node_server.py tests/test_node_bridge.py
tests/test_node_relay.py tests/test_node_bundle.py
tests/test_node_install.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py
tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict
--cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report=
--tb=short` -- 1495 passed and one minimal-node stdio response test
failed;
strict idle accounting also flagged the assembly test. Both cases pass
together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or
timing limits were weakened. The new authorization, revocation and host
compatibility tests all pass.
- The five focused cases failed on the unfixed implementation first: a
different authenticated task selecting another grant, a stream with no
task, each missing Unix socket API, and an astral Unicode chunk over the
node's byte limit. Regressions also cover revocation, grant replacement
and link replacement while an endpoint connection is opening, and a real
remote turn continuing when both host Unix socket APIs are absent.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 97.94% (381/389 changed executable lines); this fix alone is 100%
(40/40). Python lint, type, dependency
and import checks pass; all 10 import contracts hold. The scope checks
for commit messages, source language and file sizes pass. Both RPC
generators pass `--check` after integrating current main.
- A real Linux node with two concurrent tasks and a host loopback
endpoint:
before, the attacker's own task token plus the victim's public grant
names returns the victim endpoint's marker; after, it returns no data
and the endpoint is never reached. Its own grant works, unknown and
revoked grants fail, and the same attack remains denied after the relay
is killed and an ordinary ping reattaches both tasks. A full chunk of
262144 astral characters was lost/corrupted before and arrives intact
after. The final documented source tree was reinstalled and checked
again.
- Real ACP dispatch through the node's bridge reaches a host-held stdio
MCP tool and returns its marker; its grant is removed when the turn
ends.
With the host Unix socket APIs removed to simulate Windows, the remote
agent still completes, receives no MCP, explains the host limitation,
and the host MCP ledger records no extra call. This is not a native
Windows run. Dedicated test tasks, host endpoints and bridge processes
are cleaned up.
- CONTEXT.md, module/field docstrings, both collaboration guides, the
external design notes and this description were checked against the
implementation. Main's no-debug-switch behavior and full-turn recovery
documentation are retained.
Full combined-tree verification: `uv run --no-sync pytest -n 4
--tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure
reproduces on current main in a separate checkout: host CLI color
rendering,
LibreOffice detection and the two simulation cases. No other test fails.
The minimal-node response test that failed in the focused parallel
selection
passes in this full run. Diff coverage: 97.94% (381/389 executable
changed lines).
Earlier author verification:
- `uv run pytest tests/test_node_bridge.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_bundle.py tests/test_node_install.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py`
-- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge
and a stdio node naming none, the token written owner-only beside the
record, a stream opened with the task's token and refused with another
or for a task not running, refused with no host holding the task,
refused past 16 per task, one per bridge connection, the pieces of a
frame crossing both ways as they are and a stranger carrying nothing,
and every way a stream ends (the bridge closing, the host closing, the
bridge's connection ending, the host's link ending, the agent ending)
with the other end told; the bridge as a process against a real daemon
over its socket: a frame to the host and one back whole, a large frame
each way in pieces, the agent closing its stdin ending the stream and
the bridge leaving clean, the host closing the stream ending the bridge,
a wrong token and a missing daemon each refused on stderr with exit 1;
the stand-in machine with the real daemon and the real bridge: an acp
stub there that connects to its `mcpServers` and calls a tool reaches a
stdio server here and answers with its result, the stanza handed to it
naming the node's bridge, the task, the dispatch and the server and
nothing of the server's definition or socket, the ledger here showing
one call, no bridge outliving the dispatch; the reply's note and the
roster predicate for a machine row.
- `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed,
4 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`,
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`
and
`test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`,
which fail
the same way on main on this machine.
- Real machine (a registered Linux machine with Claude Code signed in,
the node reinstalled with the code before and after):
A28, before (the
code of #934's head and its node): the turn closed with "MCP servers
'probe' were not lent", Claude Code said no tool of that name was
connected, and the server here saw no call; after: `PROBE_OK:r2-lend`
in 9.4 s, the server here recorded the call, and the turn closed with
"lent to acp agent ... through the Raven node on machine ...". A34:
after
the turn the machine's process list shows no bridge, a grep of its node
directory finds nothing of the server's command or of this computer's
socket directory, and no endpoint socket is left here. A29: with the
tool switched off here, Claude Code answered UNAVAILABLE and the server
saw no call. A30: a 40 s tool call under way, the bridge visible in the
machine's process list, then cancelled: 3 s later no bridge there and
no socket here. A31: the relay ssh killed during a 20 s call: the turn
ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next
dispatch reached the server again in 7.6 s. A32: five calls in one turn
all reached the server and came back; an 8 MiB answer: the server here
produced it, and a plain ACP agent on the machine (a stdlib stub that
speaks the stanza's bridge directly) received it whole through the node
-- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude
Code itself closed the server on that answer and reported the call
failed: the agent's own cap on a tool result, not the link's.
- `ruff`, the commit-message and PR-body checks -- clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese collaboration guides describe lending,
task-bound authorization,
Windows withholding, and the existing CLI/restart limitations.
## Risk
Registered-machine support is enabled by configuration, without the
removed
trial environment switch. An ACP entry on a machine receives its granted
MCP servers on a Unix-socket-capable host; the host's normal MCP policy
still
applies. This intentionally exposes those tools to the selected remote
agent.
The server definition, credentials and endpoint paths remain on the
host.
The daemon identifies a bridge's task by the connecting process (the
peer
pid the kernel reports, walked to the agent it started), not by any
token or
name the bridge presents, so a peer task sharing the OS account cannot
open
another task's stream; the host then authorizes that task against the
dispatch/server grant. A system that reports no peer pid opens no
stream.
On Windows hosts remote lending is withheld explicitly, without aborting
the
remote task. Protocol 6 remains the MCP wire version; host and node must
also
have the same code digest, so updated nodes must be reinstalled.
Rollback is
a revert; the machine then carries no stream and receives no bridge
stanza.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#934 (merged); #920 and #922 (merged).
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-15 15:37:50 +08:00
|
|
|
|
feat: enable MCP lending on Windows hosts (#972)
## Summary
Windows hosts can lend granted MCP servers to local and machine-based
ACP agents through authenticated loopback endpoints. Hosts with Unix
sockets retain that transport. Server definitions, upstream credentials,
sandbox policy and disabled-tool enforcement stay on the host.
- Share one connector between the local bridge and node client. Bind TCP
endpoints only to 127.0.0.1 and authenticate with a fresh per-endpoint
token before opening an upstream.
- Read inherited Windows stdin pipes through a daemon worker with
backpressure. Acknowledge each chunk on the event loop before reading
another, so neither the reader buffer nor queued callbacks can grow
without bound.
- Cancel both bridge pumps on exit, allow up to one second for the
endpoint connection to flush, then abort the transport if it still holds
unsent bytes. A peer that stops reading cannot hold failure or
cancellation cleanup indefinitely.
- Resolve this installation's raven.exe, preserve task and session
isolation, hide endpoint credentials from grant representations, and
redact them in bridge diagnostics and session-start journal frames.
Remote dispatches do not receive the host's loopback token.
- Update capability reporting, CLI help, the remote ACP entry's
`machine` docstring, CONTEXT.md and the English/Chinese collaboration
guide. Add Windows CI coverage and exercise both Unix and TCP endpoints
on Linux.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Docker Linux, Debian Bookworm with procps and git, Python 3.12.12,
--init, a native Linux source copy, and a frozen environment: 783
related unit tests and all 3 real-server integration tests passed. The
source and test files were compared with the committed versions before
publication.
```sh
uv sync --frozen --extra dev
uv run --frozen --no-sync pytest tests/test_mcp_transport.py tests/test_mcp_bridge.py tests/test_mcp_endpoint.py tests/test_cli_mcp_commands.py tests/test_subagent_acp_mcp_lifecycle.py tests/test_node_client.py tests/test_acp_journal.py tests/test_subagent_acp_remote.py tests/test_subagent_manager.py tests/test_subagent_mcp_grant.py tests/test_subagent_third_party.py -q -n 4 --dist loadfile --tb=short --cov=raven --cov-report=json:/out/coverage.json --junitxml=/out/units.xml
uv run --frozen --no-sync pytest tests/integration/test_mcp_bridge_e2e.py -m integration -q -n 0 --tb=short --junitxml=/out/integration.xml
```
PR diff coverage is 98.26% (169/172 changed executable lines), above the
unchanged 90% threshold. On the host checkout, `$coverageReport` points
to the JSON exported by that Linux run:
```powershell
uv run --frozen --no-sync python scripts/coverage_gate.py --coverage-json $coverageReport diff --base-ref origin/main --head HEAD --threshold 90
```
Native Windows, Python 3.13.12, a separate frozen environment and
PYTHONUTF8=1: all 21 bridge unit tests passed; all 8 integration cases
passed, comprising 3 MCP bridge cases and 5 native command-line cases
retained in the Windows CI job.
```powershell
uv run --frozen --python 3.13 --no-sync pytest tests/test_mcp_bridge.py -q -n 0 --tb=short
uv run --frozen --python 3.13 --no-sync pytest tests/integration/test_mcp_bridge_e2e.py tests/integration/test_acp_mirasim_real_process.py -m integration -q -n 0 --tb=short
```
Seven regression cases cover paused input, resuming consumption, closing
a paused worker, a busy event loop, and teardown with pending TCP
writes: while the endpoint stays stalled, while it drains those writes,
and when a second cancel arrives after that drain finished. Aborting the
endpoint after a close that finished was wrong: the selector transport
has already run connection_lost and dropped its loop, and its
_force_close returns early only on _conn_lost, which a drained close
does not raise, so the bridge exited 1 at teardown on Linux and macOS.
The abort now depends on whether the endpoint's write buffer still holds
bytes, since a closing transport empties it before calling
connection_lost. A flag set once the close returned would miss the
second cancel, and that case fails with one. The stalled-endpoint
teardown case pauses the endpoint's reading before the frames are fed:
it used to wait for the socket to stop reading through its own flow
control, which took 1.87 s of the case's 2 s budget on Linux, so a busy
runner timed out before the teardown ran. It is now 0.17 s per case, and
20 consecutive runs under 32 busy loops on 16 cores all passed. The
original teardown and buffering cases fail without the fixes. At this
head, making the stdin worker read again without waiting for its
previous chunk to be delivered fails 3 cases, removing the close
deadline fails 2, and never aborting fails 2; aborting unconditionally
fails both drained cases. These mutations ran on a disposable copy of
the source tree, not on the committed files.
The unreachable-upstream integration test still uses its original
1.5-second deadline. Its Windows fixture bypasses the venv redirector
while preserving the venv, so an inherited stdout handle cannot mask
EOF.
Full-PR pre-commit, commitlint, commit-message, source-language, asset
and whitespace checks passed. The bridge type check passed and all 10
import contracts were kept:
```powershell
uv run --frozen --no-sync pre-commit run --from-ref origin/main --to-ref HEAD
npx --no-install commitlint --from origin/main --to HEAD --config commitlint.config.cjs
uv run --frozen --no-sync python scripts/check_commit_messages.py origin/main..HEAD
uv run --frozen --no-sync python scripts/check_large_files.py origin/main...HEAD
uv run --frozen --no-sync python scripts/check_source_language.py origin/main...HEAD
uv run --frozen --no-sync ty check raven/mcp/bridge.py
uv run --frozen --no-sync lint-imports
git diff --check origin/main...HEAD
```
These are focused local results. Remote dispatch tests use the
repository's SSH stand-in and real node processes, not an external
machine. POSIX-dependent paths are covered on Linux; no full-suite
Windows pass, live LLM run or sandbox VM run is claimed. Existing
Windows path, shell and journal portability outside this MCP change
remains out of scope.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
Windows dispatches can now use granted MCP servers. Each fallback
endpoint opens an ephemeral loopback listener with a bounded
authentication handshake and its own random token. Upstream definitions
and credentials remain on the host, and remote dispatches receive no
loopback token.
Input now pauses when a bridge cannot forward requests. During shutdown,
a connection that cannot flush within one second is aborted and its
remaining queued bytes can be discarded. This bounds cleanup when a peer
has stopped reading.
Unix sockets remain the preferred transport, and the Raven node wire
protocol is unchanged. No dependencies, configuration fields or
migrations are added. Reverting the change restores the previous Windows
MCP limitation.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Fixes #962
---------
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-19 19:58:55 +08:00
|
|
|
return address, got, SimpleNamespace(writers=writers, close=close)
|
feat(*): lend the host's MCP servers to an acp agent on a machine through its node (#947)
## Summary
Adds MCP lending for remote ACP agents on top of the machine and
recovery
services already merged in #920, #922 and #934. The branch includes
current
main and its default-on registered-machine behavior.
What this does: an acp entry on a registered machine is handed the MCP
servers its row grants, the way an entry on this computer is, and the
servers themselves stay here with their definitions and keys.
- The agent on the machine runs the node's own bridge as a stdio MCP
server (`python -I -m raven.node --mcp --dir <node dir> --task <task>
--lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte
pumps around a connection to the daemon's socket, no MCP parsed, no
credential held, the standard library only.
- The daemon carries the bridge's frames as a *stream* to the host
holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6,
`raven/node/protocol.py`, `raven/node/server.py`). The daemon tells
which
task a bridge is by the process that connected -- the peer pid off the
socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the
parent chain to the agent the daemon started, which the bridge is under
and no other task's process is; the kernel supplies that pid and a peer
cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the
task it names must be the one the connection is a process of. Two tasks
on
one machine run as one user, so no name or file a bridge could present
would tell them apart; the connecting process does. One stream per
bridge, at most 16 per task; a frame
travels in pieces of at most 256 KiB characters, the last marked
(`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the
daemon relays and keeps nothing. A stream ends with either end, the
host's link or the task, and the other end is told.
- The host opens the endpoints for a machine row exactly as for a row
here
(`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in
the host's sandbox, HTTP auth added here), binds them to the machine,
dispatch and acquired agent task
(`NodePool.lend`/`unlend`) before handing out bridge stanzas. On
`mcp/open`
the host checks the daemon-authenticated task against that binding
before
connecting; it rechecks the grant and link after the socket opens. An
unlend, replacement grant or replaced link cannot authorize that pending
connection. The host then pumps the frames both ways
(`raven/node/client.py`).
`node/hello` names the bridge; a node serving without a daemon
(`--stdio`)
names none, and the dispatch then goes without MCP and says so.
- The reply closes with the servers lent and the machine's node, or with
why none were. Remote lending requires Unix-socket support on the host:
on Windows it is withheld before endpoint setup, the task continues, and
the reply explains why. The roster's "injectable MCP" matches that
support
(`session_mcp_delivered`). The cli lane on a machine still
withholds its servers (the next change); the openai lane has no tools.
Not in this change: the cli lane's MCP file on the machine; a stream
that
survives the link going (a bridge sees its server end, the agent reports
the call failed, and the next dispatch works again); MCP for a run
picked
up after this computer restarted (#934): it reports its calls fail.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Follow-up fix 05eaba67a, after the c9de53947 review:
- The nine tests the Windows review named now pin the host fact through
one
fixture, `unix_socket_host` (`tests/conftest.py`), which sets
`raven.mcp.bridge.unix_sockets_available` to true and re-points every
`raven.*` module that imported it by name; their bodies are unchanged.
- Found complete, not just the nine: the whole suite was run with
`asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted
before collection (a pytest plugin, controller and xdist workers), at
the
merge base `6b938a426` and at `c9de53947`, and the two failure sets
compared. 17 node ids failed at the head and not at the base: the nine,
and eight this change added that run the transport for real (the bridge
module, and two lending tests through the stand-in machine). Those eight
are guarded by host capability instead of pinned: the bridge module
skips
when `sys.platform == "win32" or not unix_sockets_available()`, the two
lending tests carry the existing `skipif(not unix_sockets_available())`.
After the fix the same run fails the same 66 node ids at the head as at
the base: nothing this change adds or touches fails on a host without
unix sockets.
- A bridge whose daemon side goes ends as streams end. The host can
close a
stream while a frame of the agent's is on its way; the daemon writes
`mcp/close` and hangs up, and the frame meets the closed socket, or the
read side is reset first. That exited 1 with `BrokenPipeError` or
`ConnectionResetError`; a connection-level end is now exit 0 with
nothing
said, a frame past the cap still exits 1 with its reason, both pumps are
awaited before the socket closes, and a daemon gone before the stream
opens is one sentence, not a traceback. Four deterministic cases with a
stand-in connection fail on `c9de53947` and pass here.
- The bridge test's stand-in agent outlives its bridge, as an agent
outlives
one of its MCP servers, so the later parts of
`test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins`
no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80
here.
- The focused set (node, MCP, remote, manager and roster files) -- 935
passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
simulation cases, which fail the same way on main on this computer);
diff coverage against main 97.67% (420/430). Real machine, node
reinstalled with this tree in a directory of the check's own: a stdlib
ACP
agent's bridge reached a host-held server and 1 MiB and 8 MiB results
crossed whole, bridge exit 0; nothing was left on the machine.
Follow-up fix c9de53947, after the same-user token finding:
- The daemon no longer uses a token. `agent/launch` mints and writes
none,
`mcp.token` and the `hmac` check are gone, and `Daemon.open_stream`
identifies the bridge by the connecting process instead (`_peer_pid` +
`Daemon.task_of`). The same-user path the reviewer kept open -- a peer
reading another task's `mcp.token` and opening its stream -- is closed,
because identity is the connection the kernel reports, not a readable
secret.
- Tests, each failing on the unfixed tree first: a bridge placed under
task
B that names task A is refused (`the connection is a process of task
'agent@box-2', not 'agent@box-1'`), as are a process under no task, an
unplaced connection, and a task the daemon does not run
(`tests/test_node_server.py`); a real-OS test launches an agent, takes
its
real child, and asserts the child resolves to the task while the test's
own
process resolves to none (runs on Linux in CI); `_peer_pid` is checked
against a real socketpair; the in-process and subprocess bridge tests
now
identify by the connecting process, and a bridge this test spawns
itself,
or another task's agent spawns, never opens the stream.
- Windows withholding now covers a row on this computer too, not only a
machine row: `session_mcp_delivered` drops the `machine and` guard and
answers no whenever `unix_sockets_available()` is false, so a local acp
row is no longer advertised as injectable and no longer raises at
dispatch
on a host without the transport; `_mcp_endpoints` yields nothing before
opening an endpoint and `_mcp_note` explains it. The two tests the
Windows
review named now guard on `unix_sockets_available()`, and the
withholding
test is parameterized over a machine row and a local row.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over the node/MCP/remote suite -- 97.63% changed-line coverage
(412/422).
`ruff`, `ty` on the changed modules, and the import-cycle budget pass.
- Real machine (ravenx2, this head, node reinstalled, digest
fdbe132e76f0):
a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB
and
an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the
connecting-process identity admits the legitimate bridge end to end on
real Linux; the cross-task refusal on Linux is the real-OS test above,
which CI runs there.
Earlier review fixes (4dc5b3e1f, the task-to-grant binding). Its
host-side
binding of each dispatch's endpoints to the acquired agent task is kept;
the
daemon-side token this run verified was since replaced by the
connecting-process
identity above (c9de53947), so the token-named tests and reproduction
below no
longer describe the current tree.
- `uv run --no-sync pytest tests/test_node_client.py
tests/test_node_server.py tests/test_node_bridge.py
tests/test_node_relay.py tests/test_node_bundle.py
tests/test_node_install.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py
tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict
--cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report=
--tb=short` -- 1495 passed and one minimal-node stdio response test
failed;
strict idle accounting also flagged the assembly test. Both cases pass
together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or
timing limits were weakened. The new authorization, revocation and host
compatibility tests all pass.
- The five focused cases failed on the unfixed implementation first: a
different authenticated task selecting another grant, a stream with no
task, each missing Unix socket API, and an astral Unicode chunk over the
node's byte limit. Regressions also cover revocation, grant replacement
and link replacement while an endpoint connection is opening, and a real
remote turn continuing when both host Unix socket APIs are absent.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 97.94% (381/389 changed executable lines); this fix alone is 100%
(40/40). Python lint, type, dependency
and import checks pass; all 10 import contracts hold. The scope checks
for commit messages, source language and file sizes pass. Both RPC
generators pass `--check` after integrating current main.
- A real Linux node with two concurrent tasks and a host loopback
endpoint:
before, the attacker's own task token plus the victim's public grant
names returns the victim endpoint's marker; after, it returns no data
and the endpoint is never reached. Its own grant works, unknown and
revoked grants fail, and the same attack remains denied after the relay
is killed and an ordinary ping reattaches both tasks. A full chunk of
262144 astral characters was lost/corrupted before and arrives intact
after. The final documented source tree was reinstalled and checked
again.
- Real ACP dispatch through the node's bridge reaches a host-held stdio
MCP tool and returns its marker; its grant is removed when the turn
ends.
With the host Unix socket APIs removed to simulate Windows, the remote
agent still completes, receives no MCP, explains the host limitation,
and the host MCP ledger records no extra call. This is not a native
Windows run. Dedicated test tasks, host endpoints and bridge processes
are cleaned up.
- CONTEXT.md, module/field docstrings, both collaboration guides, the
external design notes and this description were checked against the
implementation. Main's no-debug-switch behavior and full-turn recovery
documentation are retained.
Full combined-tree verification: `uv run --no-sync pytest -n 4
--tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure
reproduces on current main in a separate checkout: host CLI color
rendering,
LibreOffice detection and the two simulation cases. No other test fails.
The minimal-node response test that failed in the focused parallel
selection
passes in this full run. Diff coverage: 97.94% (381/389 executable
changed lines).
Earlier author verification:
- `uv run pytest tests/test_node_bridge.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_bundle.py tests/test_node_install.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py`
-- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge
and a stdio node naming none, the token written owner-only beside the
record, a stream opened with the task's token and refused with another
or for a task not running, refused with no host holding the task,
refused past 16 per task, one per bridge connection, the pieces of a
frame crossing both ways as they are and a stranger carrying nothing,
and every way a stream ends (the bridge closing, the host closing, the
bridge's connection ending, the host's link ending, the agent ending)
with the other end told; the bridge as a process against a real daemon
over its socket: a frame to the host and one back whole, a large frame
each way in pieces, the agent closing its stdin ending the stream and
the bridge leaving clean, the host closing the stream ending the bridge,
a wrong token and a missing daemon each refused on stderr with exit 1;
the stand-in machine with the real daemon and the real bridge: an acp
stub there that connects to its `mcpServers` and calls a tool reaches a
stdio server here and answers with its result, the stanza handed to it
naming the node's bridge, the task, the dispatch and the server and
nothing of the server's definition or socket, the ledger here showing
one call, no bridge outliving the dispatch; the reply's note and the
roster predicate for a machine row.
- `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed,
4 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`,
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`
and
`test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`,
which fail
the same way on main on this machine.
- Real machine (a registered Linux machine with Claude Code signed in,
the node reinstalled with the code before and after):
A28, before (the
code of #934's head and its node): the turn closed with "MCP servers
'probe' were not lent", Claude Code said no tool of that name was
connected, and the server here saw no call; after: `PROBE_OK:r2-lend`
in 9.4 s, the server here recorded the call, and the turn closed with
"lent to acp agent ... through the Raven node on machine ...". A34:
after
the turn the machine's process list shows no bridge, a grep of its node
directory finds nothing of the server's command or of this computer's
socket directory, and no endpoint socket is left here. A29: with the
tool switched off here, Claude Code answered UNAVAILABLE and the server
saw no call. A30: a 40 s tool call under way, the bridge visible in the
machine's process list, then cancelled: 3 s later no bridge there and
no socket here. A31: the relay ssh killed during a 20 s call: the turn
ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next
dispatch reached the server again in 7.6 s. A32: five calls in one turn
all reached the server and came back; an 8 MiB answer: the server here
produced it, and a plain ACP agent on the machine (a stdlib stub that
speaks the stanza's bridge directly) received it whole through the node
-- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude
Code itself closed the server on that answer and reported the call
failed: the agent's own cap on a tool result, not the link's.
- `ruff`, the commit-message and PR-body checks -- clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese collaboration guides describe lending,
task-bound authorization,
Windows withholding, and the existing CLI/restart limitations.
## Risk
Registered-machine support is enabled by configuration, without the
removed
trial environment switch. An ACP entry on a machine receives its granted
MCP servers on a Unix-socket-capable host; the host's normal MCP policy
still
applies. This intentionally exposes those tools to the selected remote
agent.
The server definition, credentials and endpoint paths remain on the
host.
The daemon identifies a bridge's task by the connecting process (the
peer
pid the kernel reports, walked to the agent it started), not by any
token or
name the bridge presents, so a peer task sharing the OS account cannot
open
another task's stream; the host then authorizes that task against the
dispatch/server grant. A system that reports no peer pid opens no
stream.
On Windows hosts remote lending is withheld explicitly, without aborting
the
remote task. Protocol 6 remains the MCP wire version; host and node must
also
have the same code digest, so updated nodes must be reinstalled.
Rollback is
a revert; the machine then carries no stream and receives no bridge
stanza.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#934 (merged); #920 and #922 (merged).
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-15 15:37:50 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
def _told(client: _Client, method: str) -> list[dict[str, Any]]:
|
|
|
|
|
return [params for said, params in client.said if said == method]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_lent_server_s_stream_is_pumped_into_its_endpoint_and_the_endpoint_s_frames_go_out_in_pieces(
|
|
|
|
|
tmp_path, monkeypatch
|
|
|
|
|
):
|
|
|
|
|
monkeypatch.setattr(protocol, "MCP_CHUNK", 4)
|
|
|
|
|
pool = NodePool()
|
|
|
|
|
client = _live(pool, tmp_path)
|
|
|
|
|
node = pool._nodes["box"]
|
|
|
|
|
path, got, endpoint = await _endpoint()
|
|
|
|
|
try:
|
|
|
|
|
pool.lend("box", "d1", {"pg": path}, task="agent")
|
|
|
|
|
await pool._on_stream("box", protocol.MCP_OPEN, {"stream": 1, "task": "agent", "lend": "d1", "server": "pg"})
|
|
|
|
|
assert list(node.streams) == [1]
|
|
|
|
|
|
|
|
|
|
# The machine's pieces make one line into the endpoint.
|
|
|
|
|
await pool._on_stream("box", protocol.MCP_FRAME, {"stream": 1, "text": "abc", "end": False})
|
|
|
|
|
await pool._on_stream("box", protocol.MCP_FRAME, {"stream": 1, "text": "def", "end": True})
|
|
|
|
|
await _until(lambda: got == [b"abcdef\n"])
|
|
|
|
|
|
|
|
|
|
# The endpoint's line goes out in pieces, the last marked.
|
|
|
|
|
endpoint.writers[0].write(b"123456\n")
|
|
|
|
|
await endpoint.writers[0].drain()
|
|
|
|
|
await _until(lambda: len(_told(client, protocol.MCP_FRAME)) == 2)
|
|
|
|
|
assert _told(client, protocol.MCP_FRAME) == [
|
|
|
|
|
{"stream": 1, "text": "1234", "end": False},
|
|
|
|
|
{"stream": 1, "text": "56", "end": True},
|
|
|
|
|
]
|
|
|
|
|
|
|
|
|
|
# What is no stream's business is dropped.
|
|
|
|
|
await pool._on_stream("nobody", protocol.MCP_FRAME, {"stream": 1, "text": "x", "end": True})
|
|
|
|
|
await pool._on_stream("box", protocol.MCP_FRAME, {"stream": True, "text": "x", "end": True})
|
|
|
|
|
await pool._on_stream("box", protocol.MCP_FRAME, {"stream": 9, "text": "x", "end": True})
|
|
|
|
|
await pool._on_stream("box", protocol.MCP_FRAME, {"stream": 1, "text": 5, "end": True})
|
|
|
|
|
await asyncio.sleep(0.05)
|
|
|
|
|
assert got == [b"abcdef\n"]
|
|
|
|
|
|
|
|
|
|
# The machine closes it: nothing is told back.
|
|
|
|
|
await pool._on_stream("box", protocol.MCP_CLOSE, {"stream": 1, "reason": "the agent closed the server"})
|
|
|
|
|
assert node.streams == {} and _told(client, protocol.MCP_CLOSE) == []
|
|
|
|
|
finally:
|
|
|
|
|
endpoint.close()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_stream_the_host_cannot_serve_is_closed_with_the_reason_and_the_link_s_end_takes_the_rest(
|
|
|
|
|
tmp_path, monkeypatch
|
|
|
|
|
):
|
|
|
|
|
pool = NodePool()
|
|
|
|
|
client = _live(pool, tmp_path)
|
|
|
|
|
node = pool._nodes["box"]
|
|
|
|
|
path, got, endpoint = await _endpoint()
|
|
|
|
|
try:
|
|
|
|
|
await pool._on_stream("box", protocol.MCP_OPEN, {"stream": 1, "task": "agent", "lend": "d1", "server": "pg"})
|
|
|
|
|
assert _told(client, protocol.MCP_CLOSE) == [{"stream": 1, "reason": "server 'pg' is not lent to that run"}]
|
feat: enable MCP lending on Windows hosts (#972)
## Summary
Windows hosts can lend granted MCP servers to local and machine-based
ACP agents through authenticated loopback endpoints. Hosts with Unix
sockets retain that transport. Server definitions, upstream credentials,
sandbox policy and disabled-tool enforcement stay on the host.
- Share one connector between the local bridge and node client. Bind TCP
endpoints only to 127.0.0.1 and authenticate with a fresh per-endpoint
token before opening an upstream.
- Read inherited Windows stdin pipes through a daemon worker with
backpressure. Acknowledge each chunk on the event loop before reading
another, so neither the reader buffer nor queued callbacks can grow
without bound.
- Cancel both bridge pumps on exit, allow up to one second for the
endpoint connection to flush, then abort the transport if it still holds
unsent bytes. A peer that stops reading cannot hold failure or
cancellation cleanup indefinitely.
- Resolve this installation's raven.exe, preserve task and session
isolation, hide endpoint credentials from grant representations, and
redact them in bridge diagnostics and session-start journal frames.
Remote dispatches do not receive the host's loopback token.
- Update capability reporting, CLI help, the remote ACP entry's
`machine` docstring, CONTEXT.md and the English/Chinese collaboration
guide. Add Windows CI coverage and exercise both Unix and TCP endpoints
on Linux.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Docker Linux, Debian Bookworm with procps and git, Python 3.12.12,
--init, a native Linux source copy, and a frozen environment: 783
related unit tests and all 3 real-server integration tests passed. The
source and test files were compared with the committed versions before
publication.
```sh
uv sync --frozen --extra dev
uv run --frozen --no-sync pytest tests/test_mcp_transport.py tests/test_mcp_bridge.py tests/test_mcp_endpoint.py tests/test_cli_mcp_commands.py tests/test_subagent_acp_mcp_lifecycle.py tests/test_node_client.py tests/test_acp_journal.py tests/test_subagent_acp_remote.py tests/test_subagent_manager.py tests/test_subagent_mcp_grant.py tests/test_subagent_third_party.py -q -n 4 --dist loadfile --tb=short --cov=raven --cov-report=json:/out/coverage.json --junitxml=/out/units.xml
uv run --frozen --no-sync pytest tests/integration/test_mcp_bridge_e2e.py -m integration -q -n 0 --tb=short --junitxml=/out/integration.xml
```
PR diff coverage is 98.26% (169/172 changed executable lines), above the
unchanged 90% threshold. On the host checkout, `$coverageReport` points
to the JSON exported by that Linux run:
```powershell
uv run --frozen --no-sync python scripts/coverage_gate.py --coverage-json $coverageReport diff --base-ref origin/main --head HEAD --threshold 90
```
Native Windows, Python 3.13.12, a separate frozen environment and
PYTHONUTF8=1: all 21 bridge unit tests passed; all 8 integration cases
passed, comprising 3 MCP bridge cases and 5 native command-line cases
retained in the Windows CI job.
```powershell
uv run --frozen --python 3.13 --no-sync pytest tests/test_mcp_bridge.py -q -n 0 --tb=short
uv run --frozen --python 3.13 --no-sync pytest tests/integration/test_mcp_bridge_e2e.py tests/integration/test_acp_mirasim_real_process.py -m integration -q -n 0 --tb=short
```
Seven regression cases cover paused input, resuming consumption, closing
a paused worker, a busy event loop, and teardown with pending TCP
writes: while the endpoint stays stalled, while it drains those writes,
and when a second cancel arrives after that drain finished. Aborting the
endpoint after a close that finished was wrong: the selector transport
has already run connection_lost and dropped its loop, and its
_force_close returns early only on _conn_lost, which a drained close
does not raise, so the bridge exited 1 at teardown on Linux and macOS.
The abort now depends on whether the endpoint's write buffer still holds
bytes, since a closing transport empties it before calling
connection_lost. A flag set once the close returned would miss the
second cancel, and that case fails with one. The stalled-endpoint
teardown case pauses the endpoint's reading before the frames are fed:
it used to wait for the socket to stop reading through its own flow
control, which took 1.87 s of the case's 2 s budget on Linux, so a busy
runner timed out before the teardown ran. It is now 0.17 s per case, and
20 consecutive runs under 32 busy loops on 16 cores all passed. The
original teardown and buffering cases fail without the fixes. At this
head, making the stdin worker read again without waiting for its
previous chunk to be delivered fails 3 cases, removing the close
deadline fails 2, and never aborting fails 2; aborting unconditionally
fails both drained cases. These mutations ran on a disposable copy of
the source tree, not on the committed files.
The unreachable-upstream integration test still uses its original
1.5-second deadline. Its Windows fixture bypasses the venv redirector
while preserving the venv, so an inherited stdout handle cannot mask
EOF.
Full-PR pre-commit, commitlint, commit-message, source-language, asset
and whitespace checks passed. The bridge type check passed and all 10
import contracts were kept:
```powershell
uv run --frozen --no-sync pre-commit run --from-ref origin/main --to-ref HEAD
npx --no-install commitlint --from origin/main --to HEAD --config commitlint.config.cjs
uv run --frozen --no-sync python scripts/check_commit_messages.py origin/main..HEAD
uv run --frozen --no-sync python scripts/check_large_files.py origin/main...HEAD
uv run --frozen --no-sync python scripts/check_source_language.py origin/main...HEAD
uv run --frozen --no-sync ty check raven/mcp/bridge.py
uv run --frozen --no-sync lint-imports
git diff --check origin/main...HEAD
```
These are focused local results. Remote dispatch tests use the
repository's SSH stand-in and real node processes, not an external
machine. POSIX-dependent paths are covered on Linux; no full-suite
Windows pass, live LLM run or sandbox VM run is claimed. Existing
Windows path, shell and journal portability outside this MCP change
remains out of scope.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
Windows dispatches can now use granted MCP servers. Each fallback
endpoint opens an ephemeral loopback listener with a bounded
authentication handshake and its own random token. Upstream definitions
and credentials remain on the host, and remote dispatches receive no
loopback token.
Input now pauses when a bridge cannot forward requests. During shutdown,
a connection that cannot flush within one second is aborted and its
remaining queued bytes can be discarded. This bounds cleanup when a peer
has stopped reading.
Unix sockets remain the preferred transport, and the Raven node wire
protocol is unchanged. No dependencies, configuration fields or
migrations are added. Reverting the change restores the previous Windows
MCP limitation.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Fixes #962
---------
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-19 19:58:55 +08:00
|
|
|
pool.lend("box", "d1", {"pg": path.rsplit("/", 1)[0] + "/" + "0" * 64}, task="agent")
|
feat(*): lend the host's MCP servers to an acp agent on a machine through its node (#947)
## Summary
Adds MCP lending for remote ACP agents on top of the machine and
recovery
services already merged in #920, #922 and #934. The branch includes
current
main and its default-on registered-machine behavior.
What this does: an acp entry on a registered machine is handed the MCP
servers its row grants, the way an entry on this computer is, and the
servers themselves stay here with their definitions and keys.
- The agent on the machine runs the node's own bridge as a stdio MCP
server (`python -I -m raven.node --mcp --dir <node dir> --task <task>
--lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte
pumps around a connection to the daemon's socket, no MCP parsed, no
credential held, the standard library only.
- The daemon carries the bridge's frames as a *stream* to the host
holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6,
`raven/node/protocol.py`, `raven/node/server.py`). The daemon tells
which
task a bridge is by the process that connected -- the peer pid off the
socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the
parent chain to the agent the daemon started, which the bridge is under
and no other task's process is; the kernel supplies that pid and a peer
cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the
task it names must be the one the connection is a process of. Two tasks
on
one machine run as one user, so no name or file a bridge could present
would tell them apart; the connecting process does. One stream per
bridge, at most 16 per task; a frame
travels in pieces of at most 256 KiB characters, the last marked
(`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the
daemon relays and keeps nothing. A stream ends with either end, the
host's link or the task, and the other end is told.
- The host opens the endpoints for a machine row exactly as for a row
here
(`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in
the host's sandbox, HTTP auth added here), binds them to the machine,
dispatch and acquired agent task
(`NodePool.lend`/`unlend`) before handing out bridge stanzas. On
`mcp/open`
the host checks the daemon-authenticated task against that binding
before
connecting; it rechecks the grant and link after the socket opens. An
unlend, replacement grant or replaced link cannot authorize that pending
connection. The host then pumps the frames both ways
(`raven/node/client.py`).
`node/hello` names the bridge; a node serving without a daemon
(`--stdio`)
names none, and the dispatch then goes without MCP and says so.
- The reply closes with the servers lent and the machine's node, or with
why none were. Remote lending requires Unix-socket support on the host:
on Windows it is withheld before endpoint setup, the task continues, and
the reply explains why. The roster's "injectable MCP" matches that
support
(`session_mcp_delivered`). The cli lane on a machine still
withholds its servers (the next change); the openai lane has no tools.
Not in this change: the cli lane's MCP file on the machine; a stream
that
survives the link going (a bridge sees its server end, the agent reports
the call failed, and the next dispatch works again); MCP for a run
picked
up after this computer restarted (#934): it reports its calls fail.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Follow-up fix 05eaba67a, after the c9de53947 review:
- The nine tests the Windows review named now pin the host fact through
one
fixture, `unix_socket_host` (`tests/conftest.py`), which sets
`raven.mcp.bridge.unix_sockets_available` to true and re-points every
`raven.*` module that imported it by name; their bodies are unchanged.
- Found complete, not just the nine: the whole suite was run with
`asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted
before collection (a pytest plugin, controller and xdist workers), at
the
merge base `6b938a426` and at `c9de53947`, and the two failure sets
compared. 17 node ids failed at the head and not at the base: the nine,
and eight this change added that run the transport for real (the bridge
module, and two lending tests through the stand-in machine). Those eight
are guarded by host capability instead of pinned: the bridge module
skips
when `sys.platform == "win32" or not unix_sockets_available()`, the two
lending tests carry the existing `skipif(not unix_sockets_available())`.
After the fix the same run fails the same 66 node ids at the head as at
the base: nothing this change adds or touches fails on a host without
unix sockets.
- A bridge whose daemon side goes ends as streams end. The host can
close a
stream while a frame of the agent's is on its way; the daemon writes
`mcp/close` and hangs up, and the frame meets the closed socket, or the
read side is reset first. That exited 1 with `BrokenPipeError` or
`ConnectionResetError`; a connection-level end is now exit 0 with
nothing
said, a frame past the cap still exits 1 with its reason, both pumps are
awaited before the socket closes, and a daemon gone before the stream
opens is one sentence, not a traceback. Four deterministic cases with a
stand-in connection fail on `c9de53947` and pass here.
- The bridge test's stand-in agent outlives its bridge, as an agent
outlives
one of its MCP servers, so the later parts of
`test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins`
no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80
here.
- The focused set (node, MCP, remote, manager and roster files) -- 935
passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
simulation cases, which fail the same way on main on this computer);
diff coverage against main 97.67% (420/430). Real machine, node
reinstalled with this tree in a directory of the check's own: a stdlib
ACP
agent's bridge reached a host-held server and 1 MiB and 8 MiB results
crossed whole, bridge exit 0; nothing was left on the machine.
Follow-up fix c9de53947, after the same-user token finding:
- The daemon no longer uses a token. `agent/launch` mints and writes
none,
`mcp.token` and the `hmac` check are gone, and `Daemon.open_stream`
identifies the bridge by the connecting process instead (`_peer_pid` +
`Daemon.task_of`). The same-user path the reviewer kept open -- a peer
reading another task's `mcp.token` and opening its stream -- is closed,
because identity is the connection the kernel reports, not a readable
secret.
- Tests, each failing on the unfixed tree first: a bridge placed under
task
B that names task A is refused (`the connection is a process of task
'agent@box-2', not 'agent@box-1'`), as are a process under no task, an
unplaced connection, and a task the daemon does not run
(`tests/test_node_server.py`); a real-OS test launches an agent, takes
its
real child, and asserts the child resolves to the task while the test's
own
process resolves to none (runs on Linux in CI); `_peer_pid` is checked
against a real socketpair; the in-process and subprocess bridge tests
now
identify by the connecting process, and a bridge this test spawns
itself,
or another task's agent spawns, never opens the stream.
- Windows withholding now covers a row on this computer too, not only a
machine row: `session_mcp_delivered` drops the `machine and` guard and
answers no whenever `unix_sockets_available()` is false, so a local acp
row is no longer advertised as injectable and no longer raises at
dispatch
on a host without the transport; `_mcp_endpoints` yields nothing before
opening an endpoint and `_mcp_note` explains it. The two tests the
Windows
review named now guard on `unix_sockets_available()`, and the
withholding
test is parameterized over a machine row and a local row.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over the node/MCP/remote suite -- 97.63% changed-line coverage
(412/422).
`ruff`, `ty` on the changed modules, and the import-cycle budget pass.
- Real machine (ravenx2, this head, node reinstalled, digest
fdbe132e76f0):
a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB
and
an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the
connecting-process identity admits the legitimate bridge end to end on
real Linux; the cross-task refusal on Linux is the real-OS test above,
which CI runs there.
Earlier review fixes (4dc5b3e1f, the task-to-grant binding). Its
host-side
binding of each dispatch's endpoints to the acquired agent task is kept;
the
daemon-side token this run verified was since replaced by the
connecting-process
identity above (c9de53947), so the token-named tests and reproduction
below no
longer describe the current tree.
- `uv run --no-sync pytest tests/test_node_client.py
tests/test_node_server.py tests/test_node_bridge.py
tests/test_node_relay.py tests/test_node_bundle.py
tests/test_node_install.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py
tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict
--cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report=
--tb=short` -- 1495 passed and one minimal-node stdio response test
failed;
strict idle accounting also flagged the assembly test. Both cases pass
together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or
timing limits were weakened. The new authorization, revocation and host
compatibility tests all pass.
- The five focused cases failed on the unfixed implementation first: a
different authenticated task selecting another grant, a stream with no
task, each missing Unix socket API, and an astral Unicode chunk over the
node's byte limit. Regressions also cover revocation, grant replacement
and link replacement while an endpoint connection is opening, and a real
remote turn continuing when both host Unix socket APIs are absent.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 97.94% (381/389 changed executable lines); this fix alone is 100%
(40/40). Python lint, type, dependency
and import checks pass; all 10 import contracts hold. The scope checks
for commit messages, source language and file sizes pass. Both RPC
generators pass `--check` after integrating current main.
- A real Linux node with two concurrent tasks and a host loopback
endpoint:
before, the attacker's own task token plus the victim's public grant
names returns the victim endpoint's marker; after, it returns no data
and the endpoint is never reached. Its own grant works, unknown and
revoked grants fail, and the same attack remains denied after the relay
is killed and an ordinary ping reattaches both tasks. A full chunk of
262144 astral characters was lost/corrupted before and arrives intact
after. The final documented source tree was reinstalled and checked
again.
- Real ACP dispatch through the node's bridge reaches a host-held stdio
MCP tool and returns its marker; its grant is removed when the turn
ends.
With the host Unix socket APIs removed to simulate Windows, the remote
agent still completes, receives no MCP, explains the host limitation,
and the host MCP ledger records no extra call. This is not a native
Windows run. Dedicated test tasks, host endpoints and bridge processes
are cleaned up.
- CONTEXT.md, module/field docstrings, both collaboration guides, the
external design notes and this description were checked against the
implementation. Main's no-debug-switch behavior and full-turn recovery
documentation are retained.
Full combined-tree verification: `uv run --no-sync pytest -n 4
--tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure
reproduces on current main in a separate checkout: host CLI color
rendering,
LibreOffice detection and the two simulation cases. No other test fails.
The minimal-node response test that failed in the focused parallel
selection
passes in this full run. Diff coverage: 97.94% (381/389 executable
changed lines).
Earlier author verification:
- `uv run pytest tests/test_node_bridge.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_bundle.py tests/test_node_install.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py`
-- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge
and a stdio node naming none, the token written owner-only beside the
record, a stream opened with the task's token and refused with another
or for a task not running, refused with no host holding the task,
refused past 16 per task, one per bridge connection, the pieces of a
frame crossing both ways as they are and a stranger carrying nothing,
and every way a stream ends (the bridge closing, the host closing, the
bridge's connection ending, the host's link ending, the agent ending)
with the other end told; the bridge as a process against a real daemon
over its socket: a frame to the host and one back whole, a large frame
each way in pieces, the agent closing its stdin ending the stream and
the bridge leaving clean, the host closing the stream ending the bridge,
a wrong token and a missing daemon each refused on stderr with exit 1;
the stand-in machine with the real daemon and the real bridge: an acp
stub there that connects to its `mcpServers` and calls a tool reaches a
stdio server here and answers with its result, the stanza handed to it
naming the node's bridge, the task, the dispatch and the server and
nothing of the server's definition or socket, the ledger here showing
one call, no bridge outliving the dispatch; the reply's note and the
roster predicate for a machine row.
- `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed,
4 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`,
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`
and
`test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`,
which fail
the same way on main on this machine.
- Real machine (a registered Linux machine with Claude Code signed in,
the node reinstalled with the code before and after):
A28, before (the
code of #934's head and its node): the turn closed with "MCP servers
'probe' were not lent", Claude Code said no tool of that name was
connected, and the server here saw no call; after: `PROBE_OK:r2-lend`
in 9.4 s, the server here recorded the call, and the turn closed with
"lent to acp agent ... through the Raven node on machine ...". A34:
after
the turn the machine's process list shows no bridge, a grep of its node
directory finds nothing of the server's command or of this computer's
socket directory, and no endpoint socket is left here. A29: with the
tool switched off here, Claude Code answered UNAVAILABLE and the server
saw no call. A30: a 40 s tool call under way, the bridge visible in the
machine's process list, then cancelled: 3 s later no bridge there and
no socket here. A31: the relay ssh killed during a 20 s call: the turn
ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next
dispatch reached the server again in 7.6 s. A32: five calls in one turn
all reached the server and came back; an 8 MiB answer: the server here
produced it, and a plain ACP agent on the machine (a stdlib stub that
speaks the stanza's bridge directly) received it whole through the node
-- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude
Code itself closed the server on that answer and reported the call
failed: the agent's own cap on a tool result, not the link's.
- `ruff`, the commit-message and PR-body checks -- clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese collaboration guides describe lending,
task-bound authorization,
Windows withholding, and the existing CLI/restart limitations.
## Risk
Registered-machine support is enabled by configuration, without the
removed
trial environment switch. An ACP entry on a machine receives its granted
MCP servers on a Unix-socket-capable host; the host's normal MCP policy
still
applies. This intentionally exposes those tools to the selected remote
agent.
The server definition, credentials and endpoint paths remain on the
host.
The daemon identifies a bridge's task by the connecting process (the
peer
pid the kernel reports, walked to the agent it started), not by any
token or
name the bridge presents, so a peer task sharing the OS account cannot
open
another task's stream; the host then authorizes that task against the
dispatch/server grant. A system that reports no peer pid opens no
stream.
On Windows hosts remote lending is withheld explicitly, without aborting
the
remote task. Protocol 6 remains the MCP wire version; host and node must
also
have the same code digest, so updated nodes must be reinstalled.
Rollback is
a revert; the machine then carries no stream and receives no bridge
stanza.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#934 (merged); #920 and #922 (merged).
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-15 15:37:50 +08:00
|
|
|
await pool._on_stream("box", protocol.MCP_OPEN, {"stream": 2, "task": "agent", "lend": "d1", "server": "pg"})
|
|
|
|
|
gone = _told(client, protocol.MCP_CLOSE)[-1]
|
|
|
|
|
assert gone["stream"] == 2 and gone["reason"].startswith("the host's endpoint for 'pg' is gone: ")
|
|
|
|
|
assert node.streams == {}
|
|
|
|
|
|
|
|
|
|
pool.lend("box", "d1", {"pg": path}, task="agent")
|
|
|
|
|
monkeypatch.setattr(protocol, "MCP_MAX_FRAME", 8)
|
|
|
|
|
# The machine's frame past the cap.
|
|
|
|
|
await pool._on_stream("box", protocol.MCP_OPEN, {"stream": 3, "task": "agent", "lend": "d1", "server": "pg"})
|
|
|
|
|
await pool._on_stream("box", protocol.MCP_FRAME, {"stream": 3, "text": "123456789", "end": False})
|
|
|
|
|
assert 3 not in node.streams
|
|
|
|
|
assert _told(client, protocol.MCP_CLOSE)[-1] == {"stream": 3, "reason": "a frame exceeds 8 bytes"}
|
|
|
|
|
# The endpoint's frame past the cap.
|
|
|
|
|
await pool._on_stream("box", protocol.MCP_OPEN, {"stream": 4, "task": "agent", "lend": "d1", "server": "pg"})
|
|
|
|
|
await _until(lambda: len(endpoint.writers) == 2)
|
|
|
|
|
endpoint.writers[-1].write(b"0123456789\n")
|
|
|
|
|
await endpoint.writers[-1].drain()
|
|
|
|
|
await _until(lambda: 4 not in node.streams)
|
|
|
|
|
assert _told(client, protocol.MCP_CLOSE)[-1] == {"stream": 4, "reason": "frame exceeds 8 bytes"}
|
|
|
|
|
monkeypatch.setattr(protocol, "MCP_MAX_FRAME", 32 * 1024 * 1024)
|
|
|
|
|
|
|
|
|
|
# The endpoint closed under the host's write.
|
|
|
|
|
await pool._on_stream("box", protocol.MCP_OPEN, {"stream": 5, "task": "agent", "lend": "d1", "server": "pg"})
|
|
|
|
|
stream = node.streams[5]
|
|
|
|
|
real, stream.writer = (
|
|
|
|
|
stream.writer,
|
|
|
|
|
SimpleNamespace(
|
|
|
|
|
write=lambda data: (_ for _ in ()).throw(ConnectionResetError("closed")), close=lambda: None
|
|
|
|
|
),
|
|
|
|
|
)
|
|
|
|
|
await pool._on_stream("box", protocol.MCP_FRAME, {"stream": 5, "text": "x", "end": True})
|
|
|
|
|
real.close()
|
|
|
|
|
assert 5 not in node.streams
|
|
|
|
|
assert _told(client, protocol.MCP_CLOSE)[-1] == {"stream": 5, "reason": "the host's endpoint closed"}
|
|
|
|
|
|
|
|
|
|
# The link is gone while the endpoint speaks: the stream goes, and nothing can be told.
|
|
|
|
|
await pool._on_stream("box", protocol.MCP_OPEN, {"stream": 6, "task": "agent", "lend": "d1", "server": "pg"})
|
|
|
|
|
told_before = len(client.said)
|
|
|
|
|
|
|
|
|
|
async def dead(method: str, params: dict[str, Any]) -> None:
|
|
|
|
|
raise AcpConnectionError("the link is gone")
|
|
|
|
|
|
|
|
|
|
client.notify = dead # type: ignore[method-assign]
|
|
|
|
|
await _until(lambda: len(endpoint.writers) == 4)
|
|
|
|
|
endpoint.writers[-1].write(b"hello\n")
|
|
|
|
|
await endpoint.writers[-1].drain()
|
|
|
|
|
await _until(lambda: 6 not in node.streams)
|
|
|
|
|
assert len(client.said) == told_before
|
|
|
|
|
client.notify = _Client.notify.__get__(client) # type: ignore[method-assign]
|
|
|
|
|
|
|
|
|
|
# The dispatch ends: its streams are told; another dispatch's stay. A machine with no link is nothing to unlend.
|
|
|
|
|
await pool._on_stream("box", protocol.MCP_OPEN, {"stream": 7, "task": "agent", "lend": "d1", "server": "pg"})
|
|
|
|
|
pool.lend("box", "d2", {"pg": path}, task="agent")
|
|
|
|
|
await pool._on_stream("box", protocol.MCP_OPEN, {"stream": 8, "task": "agent", "lend": "d2", "server": "pg"})
|
|
|
|
|
await pool.unlend("box", "d1")
|
|
|
|
|
assert list(node.streams) == [8]
|
|
|
|
|
assert _told(client, protocol.MCP_CLOSE)[-1] == {"stream": 7, "reason": "the dispatch ended"}
|
|
|
|
|
await pool.unlend("nobody", "d1")
|
|
|
|
|
|
|
|
|
|
# The link's end takes the rest, with no word down a link that is gone.
|
|
|
|
|
await pool._drop("box")
|
|
|
|
|
assert node.streams == {} and _told(client, protocol.MCP_CLOSE)[-1]["stream"] == 7
|
|
|
|
|
finally:
|
|
|
|
|
endpoint.close()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_the_bridge_argv_is_the_node_s_own_and_a_node_without_a_daemon_names_none(tmp_path):
|
|
|
|
|
pool = NodePool()
|
|
|
|
|
_live(pool, tmp_path)
|
|
|
|
|
assert await pool.bridge_argv("box") is None
|
|
|
|
|
|
|
|
|
|
pool._nodes["box"].hello["bridge"] = ["/opt/node/bin/python", "-I", "-m", "raven.node", "--mcp", "--dir", "/n"]
|
|
|
|
|
|
|
|
|
|
assert await pool.bridge_argv("box") == ["/opt/node/bin/python", "-I", "-m", "raven.node", "--mcp", "--dir", "/n"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_an_mcp_stream_cannot_claim_another_tasks_lend(tmp_path):
|
|
|
|
|
pool = NodePool()
|
|
|
|
|
client = _live(pool, tmp_path)
|
|
|
|
|
node = pool._nodes["box"]
|
|
|
|
|
path, got, endpoint = await _endpoint()
|
|
|
|
|
try:
|
|
|
|
|
pool.lend("box", "victim-dispatch", {"pg": path}, task="victim")
|
|
|
|
|
pool.lend("box", "own-dispatch", {"pg": path}, task="attacker")
|
|
|
|
|
await pool._on_stream(
|
|
|
|
|
"box", protocol.MCP_OPEN, {"stream": 1, "task": "attacker", "lend": "victim-dispatch", "server": "pg"}
|
|
|
|
|
)
|
|
|
|
|
assert 1 not in node.streams, "a valid task must not select a different task's host endpoint"
|
|
|
|
|
assert not endpoint.writers and not got, "authorization precedes the host connection"
|
|
|
|
|
await pool._on_stream(
|
|
|
|
|
"box", protocol.MCP_OPEN, {"stream": 2, "task": "attacker", "lend": "own-dispatch", "server": "pg"}
|
|
|
|
|
)
|
|
|
|
|
assert 2 in node.streams, "the same authenticated task may open its own grant"
|
|
|
|
|
await pool.unlend("box", "own-dispatch")
|
|
|
|
|
finally:
|
|
|
|
|
await pool._drop("box")
|
|
|
|
|
endpoint.close()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_an_mcp_stream_without_a_task_is_refused_before_connecting(tmp_path, monkeypatch):
|
|
|
|
|
pool = NodePool()
|
|
|
|
|
client = _live(pool, tmp_path)
|
|
|
|
|
pool.lend("box", "d1", {"pg": "/unused"}, task="agent")
|
|
|
|
|
|
|
|
|
|
async def connected(*args, **kwargs):
|
|
|
|
|
pytest.fail("a missing task reached the connection")
|
|
|
|
|
|
feat: enable MCP lending on Windows hosts (#972)
## Summary
Windows hosts can lend granted MCP servers to local and machine-based
ACP agents through authenticated loopback endpoints. Hosts with Unix
sockets retain that transport. Server definitions, upstream credentials,
sandbox policy and disabled-tool enforcement stay on the host.
- Share one connector between the local bridge and node client. Bind TCP
endpoints only to 127.0.0.1 and authenticate with a fresh per-endpoint
token before opening an upstream.
- Read inherited Windows stdin pipes through a daemon worker with
backpressure. Acknowledge each chunk on the event loop before reading
another, so neither the reader buffer nor queued callbacks can grow
without bound.
- Cancel both bridge pumps on exit, allow up to one second for the
endpoint connection to flush, then abort the transport if it still holds
unsent bytes. A peer that stops reading cannot hold failure or
cancellation cleanup indefinitely.
- Resolve this installation's raven.exe, preserve task and session
isolation, hide endpoint credentials from grant representations, and
redact them in bridge diagnostics and session-start journal frames.
Remote dispatches do not receive the host's loopback token.
- Update capability reporting, CLI help, the remote ACP entry's
`machine` docstring, CONTEXT.md and the English/Chinese collaboration
guide. Add Windows CI coverage and exercise both Unix and TCP endpoints
on Linux.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Docker Linux, Debian Bookworm with procps and git, Python 3.12.12,
--init, a native Linux source copy, and a frozen environment: 783
related unit tests and all 3 real-server integration tests passed. The
source and test files were compared with the committed versions before
publication.
```sh
uv sync --frozen --extra dev
uv run --frozen --no-sync pytest tests/test_mcp_transport.py tests/test_mcp_bridge.py tests/test_mcp_endpoint.py tests/test_cli_mcp_commands.py tests/test_subagent_acp_mcp_lifecycle.py tests/test_node_client.py tests/test_acp_journal.py tests/test_subagent_acp_remote.py tests/test_subagent_manager.py tests/test_subagent_mcp_grant.py tests/test_subagent_third_party.py -q -n 4 --dist loadfile --tb=short --cov=raven --cov-report=json:/out/coverage.json --junitxml=/out/units.xml
uv run --frozen --no-sync pytest tests/integration/test_mcp_bridge_e2e.py -m integration -q -n 0 --tb=short --junitxml=/out/integration.xml
```
PR diff coverage is 98.26% (169/172 changed executable lines), above the
unchanged 90% threshold. On the host checkout, `$coverageReport` points
to the JSON exported by that Linux run:
```powershell
uv run --frozen --no-sync python scripts/coverage_gate.py --coverage-json $coverageReport diff --base-ref origin/main --head HEAD --threshold 90
```
Native Windows, Python 3.13.12, a separate frozen environment and
PYTHONUTF8=1: all 21 bridge unit tests passed; all 8 integration cases
passed, comprising 3 MCP bridge cases and 5 native command-line cases
retained in the Windows CI job.
```powershell
uv run --frozen --python 3.13 --no-sync pytest tests/test_mcp_bridge.py -q -n 0 --tb=short
uv run --frozen --python 3.13 --no-sync pytest tests/integration/test_mcp_bridge_e2e.py tests/integration/test_acp_mirasim_real_process.py -m integration -q -n 0 --tb=short
```
Seven regression cases cover paused input, resuming consumption, closing
a paused worker, a busy event loop, and teardown with pending TCP
writes: while the endpoint stays stalled, while it drains those writes,
and when a second cancel arrives after that drain finished. Aborting the
endpoint after a close that finished was wrong: the selector transport
has already run connection_lost and dropped its loop, and its
_force_close returns early only on _conn_lost, which a drained close
does not raise, so the bridge exited 1 at teardown on Linux and macOS.
The abort now depends on whether the endpoint's write buffer still holds
bytes, since a closing transport empties it before calling
connection_lost. A flag set once the close returned would miss the
second cancel, and that case fails with one. The stalled-endpoint
teardown case pauses the endpoint's reading before the frames are fed:
it used to wait for the socket to stop reading through its own flow
control, which took 1.87 s of the case's 2 s budget on Linux, so a busy
runner timed out before the teardown ran. It is now 0.17 s per case, and
20 consecutive runs under 32 busy loops on 16 cores all passed. The
original teardown and buffering cases fail without the fixes. At this
head, making the stdin worker read again without waiting for its
previous chunk to be delivered fails 3 cases, removing the close
deadline fails 2, and never aborting fails 2; aborting unconditionally
fails both drained cases. These mutations ran on a disposable copy of
the source tree, not on the committed files.
The unreachable-upstream integration test still uses its original
1.5-second deadline. Its Windows fixture bypasses the venv redirector
while preserving the venv, so an inherited stdout handle cannot mask
EOF.
Full-PR pre-commit, commitlint, commit-message, source-language, asset
and whitespace checks passed. The bridge type check passed and all 10
import contracts were kept:
```powershell
uv run --frozen --no-sync pre-commit run --from-ref origin/main --to-ref HEAD
npx --no-install commitlint --from origin/main --to HEAD --config commitlint.config.cjs
uv run --frozen --no-sync python scripts/check_commit_messages.py origin/main..HEAD
uv run --frozen --no-sync python scripts/check_large_files.py origin/main...HEAD
uv run --frozen --no-sync python scripts/check_source_language.py origin/main...HEAD
uv run --frozen --no-sync ty check raven/mcp/bridge.py
uv run --frozen --no-sync lint-imports
git diff --check origin/main...HEAD
```
These are focused local results. Remote dispatch tests use the
repository's SSH stand-in and real node processes, not an external
machine. POSIX-dependent paths are covered on Linux; no full-suite
Windows pass, live LLM run or sandbox VM run is claimed. Existing
Windows path, shell and journal portability outside this MCP change
remains out of scope.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
Windows dispatches can now use granted MCP servers. Each fallback
endpoint opens an ephemeral loopback listener with a bounded
authentication handshake and its own random token. Upstream definitions
and credentials remain on the host, and remote dispatches receive no
loopback token.
Input now pauses when a bridge cannot forward requests. During shutdown,
a connection that cannot flush within one second is aborted and its
remaining queued bytes can be discarded. This bounds cleanup when a peer
has stopped reading.
Unix sockets remain the preferred transport, and the Raven node wire
protocol is unchanged. No dependencies, configuration fields or
migrations are added. Reverting the change restores the previous Windows
MCP limitation.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Fixes #962
---------
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-19 19:58:55 +08:00
|
|
|
monkeypatch.setattr(node_client, "open_endpoint", connected)
|
feat(*): lend the host's MCP servers to an acp agent on a machine through its node (#947)
## Summary
Adds MCP lending for remote ACP agents on top of the machine and
recovery
services already merged in #920, #922 and #934. The branch includes
current
main and its default-on registered-machine behavior.
What this does: an acp entry on a registered machine is handed the MCP
servers its row grants, the way an entry on this computer is, and the
servers themselves stay here with their definitions and keys.
- The agent on the machine runs the node's own bridge as a stdio MCP
server (`python -I -m raven.node --mcp --dir <node dir> --task <task>
--lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte
pumps around a connection to the daemon's socket, no MCP parsed, no
credential held, the standard library only.
- The daemon carries the bridge's frames as a *stream* to the host
holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6,
`raven/node/protocol.py`, `raven/node/server.py`). The daemon tells
which
task a bridge is by the process that connected -- the peer pid off the
socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the
parent chain to the agent the daemon started, which the bridge is under
and no other task's process is; the kernel supplies that pid and a peer
cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the
task it names must be the one the connection is a process of. Two tasks
on
one machine run as one user, so no name or file a bridge could present
would tell them apart; the connecting process does. One stream per
bridge, at most 16 per task; a frame
travels in pieces of at most 256 KiB characters, the last marked
(`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the
daemon relays and keeps nothing. A stream ends with either end, the
host's link or the task, and the other end is told.
- The host opens the endpoints for a machine row exactly as for a row
here
(`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in
the host's sandbox, HTTP auth added here), binds them to the machine,
dispatch and acquired agent task
(`NodePool.lend`/`unlend`) before handing out bridge stanzas. On
`mcp/open`
the host checks the daemon-authenticated task against that binding
before
connecting; it rechecks the grant and link after the socket opens. An
unlend, replacement grant or replaced link cannot authorize that pending
connection. The host then pumps the frames both ways
(`raven/node/client.py`).
`node/hello` names the bridge; a node serving without a daemon
(`--stdio`)
names none, and the dispatch then goes without MCP and says so.
- The reply closes with the servers lent and the machine's node, or with
why none were. Remote lending requires Unix-socket support on the host:
on Windows it is withheld before endpoint setup, the task continues, and
the reply explains why. The roster's "injectable MCP" matches that
support
(`session_mcp_delivered`). The cli lane on a machine still
withholds its servers (the next change); the openai lane has no tools.
Not in this change: the cli lane's MCP file on the machine; a stream
that
survives the link going (a bridge sees its server end, the agent reports
the call failed, and the next dispatch works again); MCP for a run
picked
up after this computer restarted (#934): it reports its calls fail.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Follow-up fix 05eaba67a, after the c9de53947 review:
- The nine tests the Windows review named now pin the host fact through
one
fixture, `unix_socket_host` (`tests/conftest.py`), which sets
`raven.mcp.bridge.unix_sockets_available` to true and re-points every
`raven.*` module that imported it by name; their bodies are unchanged.
- Found complete, not just the nine: the whole suite was run with
`asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted
before collection (a pytest plugin, controller and xdist workers), at
the
merge base `6b938a426` and at `c9de53947`, and the two failure sets
compared. 17 node ids failed at the head and not at the base: the nine,
and eight this change added that run the transport for real (the bridge
module, and two lending tests through the stand-in machine). Those eight
are guarded by host capability instead of pinned: the bridge module
skips
when `sys.platform == "win32" or not unix_sockets_available()`, the two
lending tests carry the existing `skipif(not unix_sockets_available())`.
After the fix the same run fails the same 66 node ids at the head as at
the base: nothing this change adds or touches fails on a host without
unix sockets.
- A bridge whose daemon side goes ends as streams end. The host can
close a
stream while a frame of the agent's is on its way; the daemon writes
`mcp/close` and hangs up, and the frame meets the closed socket, or the
read side is reset first. That exited 1 with `BrokenPipeError` or
`ConnectionResetError`; a connection-level end is now exit 0 with
nothing
said, a frame past the cap still exits 1 with its reason, both pumps are
awaited before the socket closes, and a daemon gone before the stream
opens is one sentence, not a traceback. Four deterministic cases with a
stand-in connection fail on `c9de53947` and pass here.
- The bridge test's stand-in agent outlives its bridge, as an agent
outlives
one of its MCP servers, so the later parts of
`test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins`
no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80
here.
- The focused set (node, MCP, remote, manager and roster files) -- 935
passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
simulation cases, which fail the same way on main on this computer);
diff coverage against main 97.67% (420/430). Real machine, node
reinstalled with this tree in a directory of the check's own: a stdlib
ACP
agent's bridge reached a host-held server and 1 MiB and 8 MiB results
crossed whole, bridge exit 0; nothing was left on the machine.
Follow-up fix c9de53947, after the same-user token finding:
- The daemon no longer uses a token. `agent/launch` mints and writes
none,
`mcp.token` and the `hmac` check are gone, and `Daemon.open_stream`
identifies the bridge by the connecting process instead (`_peer_pid` +
`Daemon.task_of`). The same-user path the reviewer kept open -- a peer
reading another task's `mcp.token` and opening its stream -- is closed,
because identity is the connection the kernel reports, not a readable
secret.
- Tests, each failing on the unfixed tree first: a bridge placed under
task
B that names task A is refused (`the connection is a process of task
'agent@box-2', not 'agent@box-1'`), as are a process under no task, an
unplaced connection, and a task the daemon does not run
(`tests/test_node_server.py`); a real-OS test launches an agent, takes
its
real child, and asserts the child resolves to the task while the test's
own
process resolves to none (runs on Linux in CI); `_peer_pid` is checked
against a real socketpair; the in-process and subprocess bridge tests
now
identify by the connecting process, and a bridge this test spawns
itself,
or another task's agent spawns, never opens the stream.
- Windows withholding now covers a row on this computer too, not only a
machine row: `session_mcp_delivered` drops the `machine and` guard and
answers no whenever `unix_sockets_available()` is false, so a local acp
row is no longer advertised as injectable and no longer raises at
dispatch
on a host without the transport; `_mcp_endpoints` yields nothing before
opening an endpoint and `_mcp_note` explains it. The two tests the
Windows
review named now guard on `unix_sockets_available()`, and the
withholding
test is parameterized over a machine row and a local row.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over the node/MCP/remote suite -- 97.63% changed-line coverage
(412/422).
`ruff`, `ty` on the changed modules, and the import-cycle budget pass.
- Real machine (ravenx2, this head, node reinstalled, digest
fdbe132e76f0):
a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB
and
an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the
connecting-process identity admits the legitimate bridge end to end on
real Linux; the cross-task refusal on Linux is the real-OS test above,
which CI runs there.
Earlier review fixes (4dc5b3e1f, the task-to-grant binding). Its
host-side
binding of each dispatch's endpoints to the acquired agent task is kept;
the
daemon-side token this run verified was since replaced by the
connecting-process
identity above (c9de53947), so the token-named tests and reproduction
below no
longer describe the current tree.
- `uv run --no-sync pytest tests/test_node_client.py
tests/test_node_server.py tests/test_node_bridge.py
tests/test_node_relay.py tests/test_node_bundle.py
tests/test_node_install.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py
tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict
--cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report=
--tb=short` -- 1495 passed and one minimal-node stdio response test
failed;
strict idle accounting also flagged the assembly test. Both cases pass
together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or
timing limits were weakened. The new authorization, revocation and host
compatibility tests all pass.
- The five focused cases failed on the unfixed implementation first: a
different authenticated task selecting another grant, a stream with no
task, each missing Unix socket API, and an astral Unicode chunk over the
node's byte limit. Regressions also cover revocation, grant replacement
and link replacement while an endpoint connection is opening, and a real
remote turn continuing when both host Unix socket APIs are absent.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 97.94% (381/389 changed executable lines); this fix alone is 100%
(40/40). Python lint, type, dependency
and import checks pass; all 10 import contracts hold. The scope checks
for commit messages, source language and file sizes pass. Both RPC
generators pass `--check` after integrating current main.
- A real Linux node with two concurrent tasks and a host loopback
endpoint:
before, the attacker's own task token plus the victim's public grant
names returns the victim endpoint's marker; after, it returns no data
and the endpoint is never reached. Its own grant works, unknown and
revoked grants fail, and the same attack remains denied after the relay
is killed and an ordinary ping reattaches both tasks. A full chunk of
262144 astral characters was lost/corrupted before and arrives intact
after. The final documented source tree was reinstalled and checked
again.
- Real ACP dispatch through the node's bridge reaches a host-held stdio
MCP tool and returns its marker; its grant is removed when the turn
ends.
With the host Unix socket APIs removed to simulate Windows, the remote
agent still completes, receives no MCP, explains the host limitation,
and the host MCP ledger records no extra call. This is not a native
Windows run. Dedicated test tasks, host endpoints and bridge processes
are cleaned up.
- CONTEXT.md, module/field docstrings, both collaboration guides, the
external design notes and this description were checked against the
implementation. Main's no-debug-switch behavior and full-turn recovery
documentation are retained.
Full combined-tree verification: `uv run --no-sync pytest -n 4
--tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure
reproduces on current main in a separate checkout: host CLI color
rendering,
LibreOffice detection and the two simulation cases. No other test fails.
The minimal-node response test that failed in the focused parallel
selection
passes in this full run. Diff coverage: 97.94% (381/389 executable
changed lines).
Earlier author verification:
- `uv run pytest tests/test_node_bridge.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_bundle.py tests/test_node_install.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py`
-- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge
and a stdio node naming none, the token written owner-only beside the
record, a stream opened with the task's token and refused with another
or for a task not running, refused with no host holding the task,
refused past 16 per task, one per bridge connection, the pieces of a
frame crossing both ways as they are and a stranger carrying nothing,
and every way a stream ends (the bridge closing, the host closing, the
bridge's connection ending, the host's link ending, the agent ending)
with the other end told; the bridge as a process against a real daemon
over its socket: a frame to the host and one back whole, a large frame
each way in pieces, the agent closing its stdin ending the stream and
the bridge leaving clean, the host closing the stream ending the bridge,
a wrong token and a missing daemon each refused on stderr with exit 1;
the stand-in machine with the real daemon and the real bridge: an acp
stub there that connects to its `mcpServers` and calls a tool reaches a
stdio server here and answers with its result, the stanza handed to it
naming the node's bridge, the task, the dispatch and the server and
nothing of the server's definition or socket, the ledger here showing
one call, no bridge outliving the dispatch; the reply's note and the
roster predicate for a machine row.
- `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed,
4 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`,
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`
and
`test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`,
which fail
the same way on main on this machine.
- Real machine (a registered Linux machine with Claude Code signed in,
the node reinstalled with the code before and after):
A28, before (the
code of #934's head and its node): the turn closed with "MCP servers
'probe' were not lent", Claude Code said no tool of that name was
connected, and the server here saw no call; after: `PROBE_OK:r2-lend`
in 9.4 s, the server here recorded the call, and the turn closed with
"lent to acp agent ... through the Raven node on machine ...". A34:
after
the turn the machine's process list shows no bridge, a grep of its node
directory finds nothing of the server's command or of this computer's
socket directory, and no endpoint socket is left here. A29: with the
tool switched off here, Claude Code answered UNAVAILABLE and the server
saw no call. A30: a 40 s tool call under way, the bridge visible in the
machine's process list, then cancelled: 3 s later no bridge there and
no socket here. A31: the relay ssh killed during a 20 s call: the turn
ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next
dispatch reached the server again in 7.6 s. A32: five calls in one turn
all reached the server and came back; an 8 MiB answer: the server here
produced it, and a plain ACP agent on the machine (a stdlib stub that
speaks the stanza's bridge directly) received it whole through the node
-- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude
Code itself closed the server on that answer and reported the call
failed: the agent's own cap on a tool result, not the link's.
- `ruff`, the commit-message and PR-body checks -- clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese collaboration guides describe lending,
task-bound authorization,
Windows withholding, and the existing CLI/restart limitations.
## Risk
Registered-machine support is enabled by configuration, without the
removed
trial environment switch. An ACP entry on a machine receives its granted
MCP servers on a Unix-socket-capable host; the host's normal MCP policy
still
applies. This intentionally exposes those tools to the selected remote
agent.
The server definition, credentials and endpoint paths remain on the
host.
The daemon identifies a bridge's task by the connecting process (the
peer
pid the kernel reports, walked to the agent it started), not by any
token or
name the bridge presents, so a peer task sharing the OS account cannot
open
another task's stream; the host then authorizes that task against the
dispatch/server grant. A system that reports no peer pid opens no
stream.
On Windows hosts remote lending is withheld explicitly, without aborting
the
remote task. Protocol 6 remains the MCP wire version; host and node must
also
have the same code digest, so updated nodes must be reinstalled.
Rollback is
a revert; the machine then carries no stream and receives no bridge
stanza.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#934 (merged); #920 and #922 (merged).
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-15 15:37:50 +08:00
|
|
|
await pool._on_stream("box", protocol.MCP_OPEN, {"stream": 1, "lend": "d1", "server": "pg"})
|
|
|
|
|
assert _told(client, protocol.MCP_CLOSE)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize("change", ["unlend", "replace", "link"])
|
|
|
|
|
async def test_an_mcp_connection_opening_after_its_authorization_ended_is_discarded(tmp_path, monkeypatch, change):
|
|
|
|
|
pool = NodePool()
|
|
|
|
|
client = _live(pool, tmp_path)
|
|
|
|
|
node = pool._nodes["box"]
|
|
|
|
|
pool.lend("box", "d1", {"pg": "/endpoint"}, task="agent")
|
|
|
|
|
entered, resume = asyncio.Event(), asyncio.Event()
|
|
|
|
|
closed = []
|
|
|
|
|
|
|
|
|
|
async def waited():
|
|
|
|
|
pass
|
|
|
|
|
|
|
|
|
|
writer = SimpleNamespace(close=lambda: closed.append(True), wait_closed=waited)
|
|
|
|
|
|
|
|
|
|
async def opening(path):
|
|
|
|
|
assert path == "/endpoint"
|
|
|
|
|
entered.set()
|
|
|
|
|
await resume.wait()
|
|
|
|
|
return asyncio.StreamReader(), writer
|
|
|
|
|
|
feat: enable MCP lending on Windows hosts (#972)
## Summary
Windows hosts can lend granted MCP servers to local and machine-based
ACP agents through authenticated loopback endpoints. Hosts with Unix
sockets retain that transport. Server definitions, upstream credentials,
sandbox policy and disabled-tool enforcement stay on the host.
- Share one connector between the local bridge and node client. Bind TCP
endpoints only to 127.0.0.1 and authenticate with a fresh per-endpoint
token before opening an upstream.
- Read inherited Windows stdin pipes through a daemon worker with
backpressure. Acknowledge each chunk on the event loop before reading
another, so neither the reader buffer nor queued callbacks can grow
without bound.
- Cancel both bridge pumps on exit, allow up to one second for the
endpoint connection to flush, then abort the transport if it still holds
unsent bytes. A peer that stops reading cannot hold failure or
cancellation cleanup indefinitely.
- Resolve this installation's raven.exe, preserve task and session
isolation, hide endpoint credentials from grant representations, and
redact them in bridge diagnostics and session-start journal frames.
Remote dispatches do not receive the host's loopback token.
- Update capability reporting, CLI help, the remote ACP entry's
`machine` docstring, CONTEXT.md and the English/Chinese collaboration
guide. Add Windows CI coverage and exercise both Unix and TCP endpoints
on Linux.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Docker Linux, Debian Bookworm with procps and git, Python 3.12.12,
--init, a native Linux source copy, and a frozen environment: 783
related unit tests and all 3 real-server integration tests passed. The
source and test files were compared with the committed versions before
publication.
```sh
uv sync --frozen --extra dev
uv run --frozen --no-sync pytest tests/test_mcp_transport.py tests/test_mcp_bridge.py tests/test_mcp_endpoint.py tests/test_cli_mcp_commands.py tests/test_subagent_acp_mcp_lifecycle.py tests/test_node_client.py tests/test_acp_journal.py tests/test_subagent_acp_remote.py tests/test_subagent_manager.py tests/test_subagent_mcp_grant.py tests/test_subagent_third_party.py -q -n 4 --dist loadfile --tb=short --cov=raven --cov-report=json:/out/coverage.json --junitxml=/out/units.xml
uv run --frozen --no-sync pytest tests/integration/test_mcp_bridge_e2e.py -m integration -q -n 0 --tb=short --junitxml=/out/integration.xml
```
PR diff coverage is 98.26% (169/172 changed executable lines), above the
unchanged 90% threshold. On the host checkout, `$coverageReport` points
to the JSON exported by that Linux run:
```powershell
uv run --frozen --no-sync python scripts/coverage_gate.py --coverage-json $coverageReport diff --base-ref origin/main --head HEAD --threshold 90
```
Native Windows, Python 3.13.12, a separate frozen environment and
PYTHONUTF8=1: all 21 bridge unit tests passed; all 8 integration cases
passed, comprising 3 MCP bridge cases and 5 native command-line cases
retained in the Windows CI job.
```powershell
uv run --frozen --python 3.13 --no-sync pytest tests/test_mcp_bridge.py -q -n 0 --tb=short
uv run --frozen --python 3.13 --no-sync pytest tests/integration/test_mcp_bridge_e2e.py tests/integration/test_acp_mirasim_real_process.py -m integration -q -n 0 --tb=short
```
Seven regression cases cover paused input, resuming consumption, closing
a paused worker, a busy event loop, and teardown with pending TCP
writes: while the endpoint stays stalled, while it drains those writes,
and when a second cancel arrives after that drain finished. Aborting the
endpoint after a close that finished was wrong: the selector transport
has already run connection_lost and dropped its loop, and its
_force_close returns early only on _conn_lost, which a drained close
does not raise, so the bridge exited 1 at teardown on Linux and macOS.
The abort now depends on whether the endpoint's write buffer still holds
bytes, since a closing transport empties it before calling
connection_lost. A flag set once the close returned would miss the
second cancel, and that case fails with one. The stalled-endpoint
teardown case pauses the endpoint's reading before the frames are fed:
it used to wait for the socket to stop reading through its own flow
control, which took 1.87 s of the case's 2 s budget on Linux, so a busy
runner timed out before the teardown ran. It is now 0.17 s per case, and
20 consecutive runs under 32 busy loops on 16 cores all passed. The
original teardown and buffering cases fail without the fixes. At this
head, making the stdin worker read again without waiting for its
previous chunk to be delivered fails 3 cases, removing the close
deadline fails 2, and never aborting fails 2; aborting unconditionally
fails both drained cases. These mutations ran on a disposable copy of
the source tree, not on the committed files.
The unreachable-upstream integration test still uses its original
1.5-second deadline. Its Windows fixture bypasses the venv redirector
while preserving the venv, so an inherited stdout handle cannot mask
EOF.
Full-PR pre-commit, commitlint, commit-message, source-language, asset
and whitespace checks passed. The bridge type check passed and all 10
import contracts were kept:
```powershell
uv run --frozen --no-sync pre-commit run --from-ref origin/main --to-ref HEAD
npx --no-install commitlint --from origin/main --to HEAD --config commitlint.config.cjs
uv run --frozen --no-sync python scripts/check_commit_messages.py origin/main..HEAD
uv run --frozen --no-sync python scripts/check_large_files.py origin/main...HEAD
uv run --frozen --no-sync python scripts/check_source_language.py origin/main...HEAD
uv run --frozen --no-sync ty check raven/mcp/bridge.py
uv run --frozen --no-sync lint-imports
git diff --check origin/main...HEAD
```
These are focused local results. Remote dispatch tests use the
repository's SSH stand-in and real node processes, not an external
machine. POSIX-dependent paths are covered on Linux; no full-suite
Windows pass, live LLM run or sandbox VM run is claimed. Existing
Windows path, shell and journal portability outside this MCP change
remains out of scope.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
Windows dispatches can now use granted MCP servers. Each fallback
endpoint opens an ephemeral loopback listener with a bounded
authentication handshake and its own random token. Upstream definitions
and credentials remain on the host, and remote dispatches receive no
loopback token.
Input now pauses when a bridge cannot forward requests. During shutdown,
a connection that cannot flush within one second is aborted and its
remaining queued bytes can be discarded. This bounds cleanup when a peer
has stopped reading.
Unix sockets remain the preferred transport, and the Raven node wire
protocol is unchanged. No dependencies, configuration fields or
migrations are added. Reverting the change restores the previous Windows
MCP limitation.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Fixes #962
---------
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-19 19:58:55 +08:00
|
|
|
monkeypatch.setattr(node_client, "open_endpoint", opening)
|
feat(*): lend the host's MCP servers to an acp agent on a machine through its node (#947)
## Summary
Adds MCP lending for remote ACP agents on top of the machine and
recovery
services already merged in #920, #922 and #934. The branch includes
current
main and its default-on registered-machine behavior.
What this does: an acp entry on a registered machine is handed the MCP
servers its row grants, the way an entry on this computer is, and the
servers themselves stay here with their definitions and keys.
- The agent on the machine runs the node's own bridge as a stdio MCP
server (`python -I -m raven.node --mcp --dir <node dir> --task <task>
--lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte
pumps around a connection to the daemon's socket, no MCP parsed, no
credential held, the standard library only.
- The daemon carries the bridge's frames as a *stream* to the host
holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6,
`raven/node/protocol.py`, `raven/node/server.py`). The daemon tells
which
task a bridge is by the process that connected -- the peer pid off the
socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the
parent chain to the agent the daemon started, which the bridge is under
and no other task's process is; the kernel supplies that pid and a peer
cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the
task it names must be the one the connection is a process of. Two tasks
on
one machine run as one user, so no name or file a bridge could present
would tell them apart; the connecting process does. One stream per
bridge, at most 16 per task; a frame
travels in pieces of at most 256 KiB characters, the last marked
(`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the
daemon relays and keeps nothing. A stream ends with either end, the
host's link or the task, and the other end is told.
- The host opens the endpoints for a machine row exactly as for a row
here
(`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in
the host's sandbox, HTTP auth added here), binds them to the machine,
dispatch and acquired agent task
(`NodePool.lend`/`unlend`) before handing out bridge stanzas. On
`mcp/open`
the host checks the daemon-authenticated task against that binding
before
connecting; it rechecks the grant and link after the socket opens. An
unlend, replacement grant or replaced link cannot authorize that pending
connection. The host then pumps the frames both ways
(`raven/node/client.py`).
`node/hello` names the bridge; a node serving without a daemon
(`--stdio`)
names none, and the dispatch then goes without MCP and says so.
- The reply closes with the servers lent and the machine's node, or with
why none were. Remote lending requires Unix-socket support on the host:
on Windows it is withheld before endpoint setup, the task continues, and
the reply explains why. The roster's "injectable MCP" matches that
support
(`session_mcp_delivered`). The cli lane on a machine still
withholds its servers (the next change); the openai lane has no tools.
Not in this change: the cli lane's MCP file on the machine; a stream
that
survives the link going (a bridge sees its server end, the agent reports
the call failed, and the next dispatch works again); MCP for a run
picked
up after this computer restarted (#934): it reports its calls fail.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Follow-up fix 05eaba67a, after the c9de53947 review:
- The nine tests the Windows review named now pin the host fact through
one
fixture, `unix_socket_host` (`tests/conftest.py`), which sets
`raven.mcp.bridge.unix_sockets_available` to true and re-points every
`raven.*` module that imported it by name; their bodies are unchanged.
- Found complete, not just the nine: the whole suite was run with
`asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted
before collection (a pytest plugin, controller and xdist workers), at
the
merge base `6b938a426` and at `c9de53947`, and the two failure sets
compared. 17 node ids failed at the head and not at the base: the nine,
and eight this change added that run the transport for real (the bridge
module, and two lending tests through the stand-in machine). Those eight
are guarded by host capability instead of pinned: the bridge module
skips
when `sys.platform == "win32" or not unix_sockets_available()`, the two
lending tests carry the existing `skipif(not unix_sockets_available())`.
After the fix the same run fails the same 66 node ids at the head as at
the base: nothing this change adds or touches fails on a host without
unix sockets.
- A bridge whose daemon side goes ends as streams end. The host can
close a
stream while a frame of the agent's is on its way; the daemon writes
`mcp/close` and hangs up, and the frame meets the closed socket, or the
read side is reset first. That exited 1 with `BrokenPipeError` or
`ConnectionResetError`; a connection-level end is now exit 0 with
nothing
said, a frame past the cap still exits 1 with its reason, both pumps are
awaited before the socket closes, and a daemon gone before the stream
opens is one sentence, not a traceback. Four deterministic cases with a
stand-in connection fail on `c9de53947` and pass here.
- The bridge test's stand-in agent outlives its bridge, as an agent
outlives
one of its MCP servers, so the later parts of
`test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins`
no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80
here.
- The focused set (node, MCP, remote, manager and roster files) -- 935
passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
simulation cases, which fail the same way on main on this computer);
diff coverage against main 97.67% (420/430). Real machine, node
reinstalled with this tree in a directory of the check's own: a stdlib
ACP
agent's bridge reached a host-held server and 1 MiB and 8 MiB results
crossed whole, bridge exit 0; nothing was left on the machine.
Follow-up fix c9de53947, after the same-user token finding:
- The daemon no longer uses a token. `agent/launch` mints and writes
none,
`mcp.token` and the `hmac` check are gone, and `Daemon.open_stream`
identifies the bridge by the connecting process instead (`_peer_pid` +
`Daemon.task_of`). The same-user path the reviewer kept open -- a peer
reading another task's `mcp.token` and opening its stream -- is closed,
because identity is the connection the kernel reports, not a readable
secret.
- Tests, each failing on the unfixed tree first: a bridge placed under
task
B that names task A is refused (`the connection is a process of task
'agent@box-2', not 'agent@box-1'`), as are a process under no task, an
unplaced connection, and a task the daemon does not run
(`tests/test_node_server.py`); a real-OS test launches an agent, takes
its
real child, and asserts the child resolves to the task while the test's
own
process resolves to none (runs on Linux in CI); `_peer_pid` is checked
against a real socketpair; the in-process and subprocess bridge tests
now
identify by the connecting process, and a bridge this test spawns
itself,
or another task's agent spawns, never opens the stream.
- Windows withholding now covers a row on this computer too, not only a
machine row: `session_mcp_delivered` drops the `machine and` guard and
answers no whenever `unix_sockets_available()` is false, so a local acp
row is no longer advertised as injectable and no longer raises at
dispatch
on a host without the transport; `_mcp_endpoints` yields nothing before
opening an endpoint and `_mcp_note` explains it. The two tests the
Windows
review named now guard on `unix_sockets_available()`, and the
withholding
test is parameterized over a machine row and a local row.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over the node/MCP/remote suite -- 97.63% changed-line coverage
(412/422).
`ruff`, `ty` on the changed modules, and the import-cycle budget pass.
- Real machine (ravenx2, this head, node reinstalled, digest
fdbe132e76f0):
a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB
and
an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the
connecting-process identity admits the legitimate bridge end to end on
real Linux; the cross-task refusal on Linux is the real-OS test above,
which CI runs there.
Earlier review fixes (4dc5b3e1f, the task-to-grant binding). Its
host-side
binding of each dispatch's endpoints to the acquired agent task is kept;
the
daemon-side token this run verified was since replaced by the
connecting-process
identity above (c9de53947), so the token-named tests and reproduction
below no
longer describe the current tree.
- `uv run --no-sync pytest tests/test_node_client.py
tests/test_node_server.py tests/test_node_bridge.py
tests/test_node_relay.py tests/test_node_bundle.py
tests/test_node_install.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py
tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict
--cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report=
--tb=short` -- 1495 passed and one minimal-node stdio response test
failed;
strict idle accounting also flagged the assembly test. Both cases pass
together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or
timing limits were weakened. The new authorization, revocation and host
compatibility tests all pass.
- The five focused cases failed on the unfixed implementation first: a
different authenticated task selecting another grant, a stream with no
task, each missing Unix socket API, and an astral Unicode chunk over the
node's byte limit. Regressions also cover revocation, grant replacement
and link replacement while an endpoint connection is opening, and a real
remote turn continuing when both host Unix socket APIs are absent.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 97.94% (381/389 changed executable lines); this fix alone is 100%
(40/40). Python lint, type, dependency
and import checks pass; all 10 import contracts hold. The scope checks
for commit messages, source language and file sizes pass. Both RPC
generators pass `--check` after integrating current main.
- A real Linux node with two concurrent tasks and a host loopback
endpoint:
before, the attacker's own task token plus the victim's public grant
names returns the victim endpoint's marker; after, it returns no data
and the endpoint is never reached. Its own grant works, unknown and
revoked grants fail, and the same attack remains denied after the relay
is killed and an ordinary ping reattaches both tasks. A full chunk of
262144 astral characters was lost/corrupted before and arrives intact
after. The final documented source tree was reinstalled and checked
again.
- Real ACP dispatch through the node's bridge reaches a host-held stdio
MCP tool and returns its marker; its grant is removed when the turn
ends.
With the host Unix socket APIs removed to simulate Windows, the remote
agent still completes, receives no MCP, explains the host limitation,
and the host MCP ledger records no extra call. This is not a native
Windows run. Dedicated test tasks, host endpoints and bridge processes
are cleaned up.
- CONTEXT.md, module/field docstrings, both collaboration guides, the
external design notes and this description were checked against the
implementation. Main's no-debug-switch behavior and full-turn recovery
documentation are retained.
Full combined-tree verification: `uv run --no-sync pytest -n 4
--tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure
reproduces on current main in a separate checkout: host CLI color
rendering,
LibreOffice detection and the two simulation cases. No other test fails.
The minimal-node response test that failed in the focused parallel
selection
passes in this full run. Diff coverage: 97.94% (381/389 executable
changed lines).
Earlier author verification:
- `uv run pytest tests/test_node_bridge.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_bundle.py tests/test_node_install.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py`
-- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge
and a stdio node naming none, the token written owner-only beside the
record, a stream opened with the task's token and refused with another
or for a task not running, refused with no host holding the task,
refused past 16 per task, one per bridge connection, the pieces of a
frame crossing both ways as they are and a stranger carrying nothing,
and every way a stream ends (the bridge closing, the host closing, the
bridge's connection ending, the host's link ending, the agent ending)
with the other end told; the bridge as a process against a real daemon
over its socket: a frame to the host and one back whole, a large frame
each way in pieces, the agent closing its stdin ending the stream and
the bridge leaving clean, the host closing the stream ending the bridge,
a wrong token and a missing daemon each refused on stderr with exit 1;
the stand-in machine with the real daemon and the real bridge: an acp
stub there that connects to its `mcpServers` and calls a tool reaches a
stdio server here and answers with its result, the stanza handed to it
naming the node's bridge, the task, the dispatch and the server and
nothing of the server's definition or socket, the ledger here showing
one call, no bridge outliving the dispatch; the reply's note and the
roster predicate for a machine row.
- `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed,
4 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`,
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`
and
`test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`,
which fail
the same way on main on this machine.
- Real machine (a registered Linux machine with Claude Code signed in,
the node reinstalled with the code before and after):
A28, before (the
code of #934's head and its node): the turn closed with "MCP servers
'probe' were not lent", Claude Code said no tool of that name was
connected, and the server here saw no call; after: `PROBE_OK:r2-lend`
in 9.4 s, the server here recorded the call, and the turn closed with
"lent to acp agent ... through the Raven node on machine ...". A34:
after
the turn the machine's process list shows no bridge, a grep of its node
directory finds nothing of the server's command or of this computer's
socket directory, and no endpoint socket is left here. A29: with the
tool switched off here, Claude Code answered UNAVAILABLE and the server
saw no call. A30: a 40 s tool call under way, the bridge visible in the
machine's process list, then cancelled: 3 s later no bridge there and
no socket here. A31: the relay ssh killed during a 20 s call: the turn
ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next
dispatch reached the server again in 7.6 s. A32: five calls in one turn
all reached the server and came back; an 8 MiB answer: the server here
produced it, and a plain ACP agent on the machine (a stdlib stub that
speaks the stanza's bridge directly) received it whole through the node
-- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude
Code itself closed the server on that answer and reported the call
failed: the agent's own cap on a tool result, not the link's.
- `ruff`, the commit-message and PR-body checks -- clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese collaboration guides describe lending,
task-bound authorization,
Windows withholding, and the existing CLI/restart limitations.
## Risk
Registered-machine support is enabled by configuration, without the
removed
trial environment switch. An ACP entry on a machine receives its granted
MCP servers on a Unix-socket-capable host; the host's normal MCP policy
still
applies. This intentionally exposes those tools to the selected remote
agent.
The server definition, credentials and endpoint paths remain on the
host.
The daemon identifies a bridge's task by the connecting process (the
peer
pid the kernel reports, walked to the agent it started), not by any
token or
name the bridge presents, so a peer task sharing the OS account cannot
open
another task's stream; the host then authorizes that task against the
dispatch/server grant. A system that reports no peer pid opens no
stream.
On Windows hosts remote lending is withheld explicitly, without aborting
the
remote task. Protocol 6 remains the MCP wire version; host and node must
also
have the same code digest, so updated nodes must be reinstalled.
Rollback is
a revert; the machine then carries no stream and receives no bridge
stanza.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#934 (merged); #920 and #922 (merged).
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-15 15:37:50 +08:00
|
|
|
waiting = asyncio.create_task(
|
|
|
|
|
pool._on_stream("box", protocol.MCP_OPEN, {"stream": 1, "task": "agent", "lend": "d1", "server": "pg"})
|
|
|
|
|
)
|
|
|
|
|
await asyncio.wait_for(entered.wait(), 2)
|
|
|
|
|
if change == "unlend":
|
|
|
|
|
await pool.unlend("box", "d1")
|
|
|
|
|
elif change == "replace":
|
|
|
|
|
pool.lend("box", "d1", {"pg": "/other"}, task="other-agent")
|
|
|
|
|
else:
|
|
|
|
|
_live(pool, tmp_path)
|
|
|
|
|
resume.set()
|
|
|
|
|
await waiting
|
|
|
|
|
assert closed and node.streams == {}
|
|
|
|
|
assert _told(client, protocol.MCP_CLOSE)[-1]["reason"] == "the dispatch or link ended"
|
|
|
|
|
|
|
|
|
|
|
feat: enable MCP lending on Windows hosts (#972)
## Summary
Windows hosts can lend granted MCP servers to local and machine-based
ACP agents through authenticated loopback endpoints. Hosts with Unix
sockets retain that transport. Server definitions, upstream credentials,
sandbox policy and disabled-tool enforcement stay on the host.
- Share one connector between the local bridge and node client. Bind TCP
endpoints only to 127.0.0.1 and authenticate with a fresh per-endpoint
token before opening an upstream.
- Read inherited Windows stdin pipes through a daemon worker with
backpressure. Acknowledge each chunk on the event loop before reading
another, so neither the reader buffer nor queued callbacks can grow
without bound.
- Cancel both bridge pumps on exit, allow up to one second for the
endpoint connection to flush, then abort the transport if it still holds
unsent bytes. A peer that stops reading cannot hold failure or
cancellation cleanup indefinitely.
- Resolve this installation's raven.exe, preserve task and session
isolation, hide endpoint credentials from grant representations, and
redact them in bridge diagnostics and session-start journal frames.
Remote dispatches do not receive the host's loopback token.
- Update capability reporting, CLI help, the remote ACP entry's
`machine` docstring, CONTEXT.md and the English/Chinese collaboration
guide. Add Windows CI coverage and exercise both Unix and TCP endpoints
on Linux.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Docker Linux, Debian Bookworm with procps and git, Python 3.12.12,
--init, a native Linux source copy, and a frozen environment: 783
related unit tests and all 3 real-server integration tests passed. The
source and test files were compared with the committed versions before
publication.
```sh
uv sync --frozen --extra dev
uv run --frozen --no-sync pytest tests/test_mcp_transport.py tests/test_mcp_bridge.py tests/test_mcp_endpoint.py tests/test_cli_mcp_commands.py tests/test_subagent_acp_mcp_lifecycle.py tests/test_node_client.py tests/test_acp_journal.py tests/test_subagent_acp_remote.py tests/test_subagent_manager.py tests/test_subagent_mcp_grant.py tests/test_subagent_third_party.py -q -n 4 --dist loadfile --tb=short --cov=raven --cov-report=json:/out/coverage.json --junitxml=/out/units.xml
uv run --frozen --no-sync pytest tests/integration/test_mcp_bridge_e2e.py -m integration -q -n 0 --tb=short --junitxml=/out/integration.xml
```
PR diff coverage is 98.26% (169/172 changed executable lines), above the
unchanged 90% threshold. On the host checkout, `$coverageReport` points
to the JSON exported by that Linux run:
```powershell
uv run --frozen --no-sync python scripts/coverage_gate.py --coverage-json $coverageReport diff --base-ref origin/main --head HEAD --threshold 90
```
Native Windows, Python 3.13.12, a separate frozen environment and
PYTHONUTF8=1: all 21 bridge unit tests passed; all 8 integration cases
passed, comprising 3 MCP bridge cases and 5 native command-line cases
retained in the Windows CI job.
```powershell
uv run --frozen --python 3.13 --no-sync pytest tests/test_mcp_bridge.py -q -n 0 --tb=short
uv run --frozen --python 3.13 --no-sync pytest tests/integration/test_mcp_bridge_e2e.py tests/integration/test_acp_mirasim_real_process.py -m integration -q -n 0 --tb=short
```
Seven regression cases cover paused input, resuming consumption, closing
a paused worker, a busy event loop, and teardown with pending TCP
writes: while the endpoint stays stalled, while it drains those writes,
and when a second cancel arrives after that drain finished. Aborting the
endpoint after a close that finished was wrong: the selector transport
has already run connection_lost and dropped its loop, and its
_force_close returns early only on _conn_lost, which a drained close
does not raise, so the bridge exited 1 at teardown on Linux and macOS.
The abort now depends on whether the endpoint's write buffer still holds
bytes, since a closing transport empties it before calling
connection_lost. A flag set once the close returned would miss the
second cancel, and that case fails with one. The stalled-endpoint
teardown case pauses the endpoint's reading before the frames are fed:
it used to wait for the socket to stop reading through its own flow
control, which took 1.87 s of the case's 2 s budget on Linux, so a busy
runner timed out before the teardown ran. It is now 0.17 s per case, and
20 consecutive runs under 32 busy loops on 16 cores all passed. The
original teardown and buffering cases fail without the fixes. At this
head, making the stdin worker read again without waiting for its
previous chunk to be delivered fails 3 cases, removing the close
deadline fails 2, and never aborting fails 2; aborting unconditionally
fails both drained cases. These mutations ran on a disposable copy of
the source tree, not on the committed files.
The unreachable-upstream integration test still uses its original
1.5-second deadline. Its Windows fixture bypasses the venv redirector
while preserving the venv, so an inherited stdout handle cannot mask
EOF.
Full-PR pre-commit, commitlint, commit-message, source-language, asset
and whitespace checks passed. The bridge type check passed and all 10
import contracts were kept:
```powershell
uv run --frozen --no-sync pre-commit run --from-ref origin/main --to-ref HEAD
npx --no-install commitlint --from origin/main --to HEAD --config commitlint.config.cjs
uv run --frozen --no-sync python scripts/check_commit_messages.py origin/main..HEAD
uv run --frozen --no-sync python scripts/check_large_files.py origin/main...HEAD
uv run --frozen --no-sync python scripts/check_source_language.py origin/main...HEAD
uv run --frozen --no-sync ty check raven/mcp/bridge.py
uv run --frozen --no-sync lint-imports
git diff --check origin/main...HEAD
```
These are focused local results. Remote dispatch tests use the
repository's SSH stand-in and real node processes, not an external
machine. POSIX-dependent paths are covered on Linux; no full-suite
Windows pass, live LLM run or sandbox VM run is claimed. Existing
Windows path, shell and journal portability outside this MCP change
remains out of scope.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
Windows dispatches can now use granted MCP servers. Each fallback
endpoint opens an ephemeral loopback listener with a bounded
authentication handshake and its own random token. Upstream definitions
and credentials remain on the host, and remote dispatches receive no
loopback token.
Input now pauses when a bridge cannot forward requests. During shutdown,
a connection that cannot flush within one second is aborted and its
remaining queued bytes can be discarded. This bounds cleanup when a peer
has stopped reading.
Unix sockets remain the preferred transport, and the Raven node wire
protocol is unchanged. No dependencies, configuration fields or
migrations are added. Reverting the change restores the previous Windows
MCP limitation.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Fixes #962
---------
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-19 19:58:55 +08:00
|
|
|
async def test_a_host_without_unix_sockets_lends_a_real_mcp_endpoint(tmp_path, monkeypatch):
|
|
|
|
|
from raven.config.schema import MCPServerConfig
|
|
|
|
|
from raven.mcp.endpoint import McpEndpoints
|
|
|
|
|
|
|
|
|
|
monkeypatch.delattr(asyncio, "start_unix_server", raising=False)
|
|
|
|
|
monkeypatch.delattr(asyncio, "open_unix_connection", raising=False)
|
feat(*): lend the host's MCP servers to an acp agent on a machine through its node (#947)
## Summary
Adds MCP lending for remote ACP agents on top of the machine and
recovery
services already merged in #920, #922 and #934. The branch includes
current
main and its default-on registered-machine behavior.
What this does: an acp entry on a registered machine is handed the MCP
servers its row grants, the way an entry on this computer is, and the
servers themselves stay here with their definitions and keys.
- The agent on the machine runs the node's own bridge as a stdio MCP
server (`python -I -m raven.node --mcp --dir <node dir> --task <task>
--lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte
pumps around a connection to the daemon's socket, no MCP parsed, no
credential held, the standard library only.
- The daemon carries the bridge's frames as a *stream* to the host
holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6,
`raven/node/protocol.py`, `raven/node/server.py`). The daemon tells
which
task a bridge is by the process that connected -- the peer pid off the
socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the
parent chain to the agent the daemon started, which the bridge is under
and no other task's process is; the kernel supplies that pid and a peer
cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the
task it names must be the one the connection is a process of. Two tasks
on
one machine run as one user, so no name or file a bridge could present
would tell them apart; the connecting process does. One stream per
bridge, at most 16 per task; a frame
travels in pieces of at most 256 KiB characters, the last marked
(`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the
daemon relays and keeps nothing. A stream ends with either end, the
host's link or the task, and the other end is told.
- The host opens the endpoints for a machine row exactly as for a row
here
(`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in
the host's sandbox, HTTP auth added here), binds them to the machine,
dispatch and acquired agent task
(`NodePool.lend`/`unlend`) before handing out bridge stanzas. On
`mcp/open`
the host checks the daemon-authenticated task against that binding
before
connecting; it rechecks the grant and link after the socket opens. An
unlend, replacement grant or replaced link cannot authorize that pending
connection. The host then pumps the frames both ways
(`raven/node/client.py`).
`node/hello` names the bridge; a node serving without a daemon
(`--stdio`)
names none, and the dispatch then goes without MCP and says so.
- The reply closes with the servers lent and the machine's node, or with
why none were. Remote lending requires Unix-socket support on the host:
on Windows it is withheld before endpoint setup, the task continues, and
the reply explains why. The roster's "injectable MCP" matches that
support
(`session_mcp_delivered`). The cli lane on a machine still
withholds its servers (the next change); the openai lane has no tools.
Not in this change: the cli lane's MCP file on the machine; a stream
that
survives the link going (a bridge sees its server end, the agent reports
the call failed, and the next dispatch works again); MCP for a run
picked
up after this computer restarted (#934): it reports its calls fail.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Follow-up fix 05eaba67a, after the c9de53947 review:
- The nine tests the Windows review named now pin the host fact through
one
fixture, `unix_socket_host` (`tests/conftest.py`), which sets
`raven.mcp.bridge.unix_sockets_available` to true and re-points every
`raven.*` module that imported it by name; their bodies are unchanged.
- Found complete, not just the nine: the whole suite was run with
`asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted
before collection (a pytest plugin, controller and xdist workers), at
the
merge base `6b938a426` and at `c9de53947`, and the two failure sets
compared. 17 node ids failed at the head and not at the base: the nine,
and eight this change added that run the transport for real (the bridge
module, and two lending tests through the stand-in machine). Those eight
are guarded by host capability instead of pinned: the bridge module
skips
when `sys.platform == "win32" or not unix_sockets_available()`, the two
lending tests carry the existing `skipif(not unix_sockets_available())`.
After the fix the same run fails the same 66 node ids at the head as at
the base: nothing this change adds or touches fails on a host without
unix sockets.
- A bridge whose daemon side goes ends as streams end. The host can
close a
stream while a frame of the agent's is on its way; the daemon writes
`mcp/close` and hangs up, and the frame meets the closed socket, or the
read side is reset first. That exited 1 with `BrokenPipeError` or
`ConnectionResetError`; a connection-level end is now exit 0 with
nothing
said, a frame past the cap still exits 1 with its reason, both pumps are
awaited before the socket closes, and a daemon gone before the stream
opens is one sentence, not a traceback. Four deterministic cases with a
stand-in connection fail on `c9de53947` and pass here.
- The bridge test's stand-in agent outlives its bridge, as an agent
outlives
one of its MCP servers, so the later parts of
`test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins`
no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80
here.
- The focused set (node, MCP, remote, manager and roster files) -- 935
passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
simulation cases, which fail the same way on main on this computer);
diff coverage against main 97.67% (420/430). Real machine, node
reinstalled with this tree in a directory of the check's own: a stdlib
ACP
agent's bridge reached a host-held server and 1 MiB and 8 MiB results
crossed whole, bridge exit 0; nothing was left on the machine.
Follow-up fix c9de53947, after the same-user token finding:
- The daemon no longer uses a token. `agent/launch` mints and writes
none,
`mcp.token` and the `hmac` check are gone, and `Daemon.open_stream`
identifies the bridge by the connecting process instead (`_peer_pid` +
`Daemon.task_of`). The same-user path the reviewer kept open -- a peer
reading another task's `mcp.token` and opening its stream -- is closed,
because identity is the connection the kernel reports, not a readable
secret.
- Tests, each failing on the unfixed tree first: a bridge placed under
task
B that names task A is refused (`the connection is a process of task
'agent@box-2', not 'agent@box-1'`), as are a process under no task, an
unplaced connection, and a task the daemon does not run
(`tests/test_node_server.py`); a real-OS test launches an agent, takes
its
real child, and asserts the child resolves to the task while the test's
own
process resolves to none (runs on Linux in CI); `_peer_pid` is checked
against a real socketpair; the in-process and subprocess bridge tests
now
identify by the connecting process, and a bridge this test spawns
itself,
or another task's agent spawns, never opens the stream.
- Windows withholding now covers a row on this computer too, not only a
machine row: `session_mcp_delivered` drops the `machine and` guard and
answers no whenever `unix_sockets_available()` is false, so a local acp
row is no longer advertised as injectable and no longer raises at
dispatch
on a host without the transport; `_mcp_endpoints` yields nothing before
opening an endpoint and `_mcp_note` explains it. The two tests the
Windows
review named now guard on `unix_sockets_available()`, and the
withholding
test is parameterized over a machine row and a local row.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over the node/MCP/remote suite -- 97.63% changed-line coverage
(412/422).
`ruff`, `ty` on the changed modules, and the import-cycle budget pass.
- Real machine (ravenx2, this head, node reinstalled, digest
fdbe132e76f0):
a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB
and
an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the
connecting-process identity admits the legitimate bridge end to end on
real Linux; the cross-task refusal on Linux is the real-OS test above,
which CI runs there.
Earlier review fixes (4dc5b3e1f, the task-to-grant binding). Its
host-side
binding of each dispatch's endpoints to the acquired agent task is kept;
the
daemon-side token this run verified was since replaced by the
connecting-process
identity above (c9de53947), so the token-named tests and reproduction
below no
longer describe the current tree.
- `uv run --no-sync pytest tests/test_node_client.py
tests/test_node_server.py tests/test_node_bridge.py
tests/test_node_relay.py tests/test_node_bundle.py
tests/test_node_install.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py
tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict
--cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report=
--tb=short` -- 1495 passed and one minimal-node stdio response test
failed;
strict idle accounting also flagged the assembly test. Both cases pass
together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or
timing limits were weakened. The new authorization, revocation and host
compatibility tests all pass.
- The five focused cases failed on the unfixed implementation first: a
different authenticated task selecting another grant, a stream with no
task, each missing Unix socket API, and an astral Unicode chunk over the
node's byte limit. Regressions also cover revocation, grant replacement
and link replacement while an endpoint connection is opening, and a real
remote turn continuing when both host Unix socket APIs are absent.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 97.94% (381/389 changed executable lines); this fix alone is 100%
(40/40). Python lint, type, dependency
and import checks pass; all 10 import contracts hold. The scope checks
for commit messages, source language and file sizes pass. Both RPC
generators pass `--check` after integrating current main.
- A real Linux node with two concurrent tasks and a host loopback
endpoint:
before, the attacker's own task token plus the victim's public grant
names returns the victim endpoint's marker; after, it returns no data
and the endpoint is never reached. Its own grant works, unknown and
revoked grants fail, and the same attack remains denied after the relay
is killed and an ordinary ping reattaches both tasks. A full chunk of
262144 astral characters was lost/corrupted before and arrives intact
after. The final documented source tree was reinstalled and checked
again.
- Real ACP dispatch through the node's bridge reaches a host-held stdio
MCP tool and returns its marker; its grant is removed when the turn
ends.
With the host Unix socket APIs removed to simulate Windows, the remote
agent still completes, receives no MCP, explains the host limitation,
and the host MCP ledger records no extra call. This is not a native
Windows run. Dedicated test tasks, host endpoints and bridge processes
are cleaned up.
- CONTEXT.md, module/field docstrings, both collaboration guides, the
external design notes and this description were checked against the
implementation. Main's no-debug-switch behavior and full-turn recovery
documentation are retained.
Full combined-tree verification: `uv run --no-sync pytest -n 4
--tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure
reproduces on current main in a separate checkout: host CLI color
rendering,
LibreOffice detection and the two simulation cases. No other test fails.
The minimal-node response test that failed in the focused parallel
selection
passes in this full run. Diff coverage: 97.94% (381/389 executable
changed lines).
Earlier author verification:
- `uv run pytest tests/test_node_bridge.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_bundle.py tests/test_node_install.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py`
-- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge
and a stdio node naming none, the token written owner-only beside the
record, a stream opened with the task's token and refused with another
or for a task not running, refused with no host holding the task,
refused past 16 per task, one per bridge connection, the pieces of a
frame crossing both ways as they are and a stranger carrying nothing,
and every way a stream ends (the bridge closing, the host closing, the
bridge's connection ending, the host's link ending, the agent ending)
with the other end told; the bridge as a process against a real daemon
over its socket: a frame to the host and one back whole, a large frame
each way in pieces, the agent closing its stdin ending the stream and
the bridge leaving clean, the host closing the stream ending the bridge,
a wrong token and a missing daemon each refused on stderr with exit 1;
the stand-in machine with the real daemon and the real bridge: an acp
stub there that connects to its `mcpServers` and calls a tool reaches a
stdio server here and answers with its result, the stanza handed to it
naming the node's bridge, the task, the dispatch and the server and
nothing of the server's definition or socket, the ledger here showing
one call, no bridge outliving the dispatch; the reply's note and the
roster predicate for a machine row.
- `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed,
4 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`,
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`
and
`test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`,
which fail
the same way on main on this machine.
- Real machine (a registered Linux machine with Claude Code signed in,
the node reinstalled with the code before and after):
A28, before (the
code of #934's head and its node): the turn closed with "MCP servers
'probe' were not lent", Claude Code said no tool of that name was
connected, and the server here saw no call; after: `PROBE_OK:r2-lend`
in 9.4 s, the server here recorded the call, and the turn closed with
"lent to acp agent ... through the Raven node on machine ...". A34:
after
the turn the machine's process list shows no bridge, a grep of its node
directory finds nothing of the server's command or of this computer's
socket directory, and no endpoint socket is left here. A29: with the
tool switched off here, Claude Code answered UNAVAILABLE and the server
saw no call. A30: a 40 s tool call under way, the bridge visible in the
machine's process list, then cancelled: 3 s later no bridge there and
no socket here. A31: the relay ssh killed during a 20 s call: the turn
ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next
dispatch reached the server again in 7.6 s. A32: five calls in one turn
all reached the server and came back; an 8 MiB answer: the server here
produced it, and a plain ACP agent on the machine (a stdlib stub that
speaks the stanza's bridge directly) received it whole through the node
-- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude
Code itself closed the server on that answer and reported the call
failed: the agent's own cap on a tool result, not the link's.
- `ruff`, the commit-message and PR-body checks -- clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese collaboration guides describe lending,
task-bound authorization,
Windows withholding, and the existing CLI/restart limitations.
## Risk
Registered-machine support is enabled by configuration, without the
removed
trial environment switch. An ACP entry on a machine receives its granted
MCP servers on a Unix-socket-capable host; the host's normal MCP policy
still
applies. This intentionally exposes those tools to the selected remote
agent.
The server definition, credentials and endpoint paths remain on the
host.
The daemon identifies a bridge's task by the connecting process (the
peer
pid the kernel reports, walked to the agent it started), not by any
token or
name the bridge presents, so a peer task sharing the OS account cannot
open
another task's stream; the host then authorizes that task against the
dispatch/server grant. A system that reports no peer pid opens no
stream.
On Windows hosts remote lending is withheld explicitly, without aborting
the
remote task. Protocol 6 remains the MCP wire version; host and node must
also
have the same code digest, so updated nodes must be reinstalled.
Rollback is
a revert; the machine then carries no stream and receives no bridge
stanza.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#934 (merged); #920 and #922 (merged).
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-15 15:37:50 +08:00
|
|
|
pool = NodePool()
|
|
|
|
|
client = _live(pool, tmp_path)
|
feat: enable MCP lending on Windows hosts (#972)
## Summary
Windows hosts can lend granted MCP servers to local and machine-based
ACP agents through authenticated loopback endpoints. Hosts with Unix
sockets retain that transport. Server definitions, upstream credentials,
sandbox policy and disabled-tool enforcement stay on the host.
- Share one connector between the local bridge and node client. Bind TCP
endpoints only to 127.0.0.1 and authenticate with a fresh per-endpoint
token before opening an upstream.
- Read inherited Windows stdin pipes through a daemon worker with
backpressure. Acknowledge each chunk on the event loop before reading
another, so neither the reader buffer nor queued callbacks can grow
without bound.
- Cancel both bridge pumps on exit, allow up to one second for the
endpoint connection to flush, then abort the transport if it still holds
unsent bytes. A peer that stops reading cannot hold failure or
cancellation cleanup indefinitely.
- Resolve this installation's raven.exe, preserve task and session
isolation, hide endpoint credentials from grant representations, and
redact them in bridge diagnostics and session-start journal frames.
Remote dispatches do not receive the host's loopback token.
- Update capability reporting, CLI help, the remote ACP entry's
`machine` docstring, CONTEXT.md and the English/Chinese collaboration
guide. Add Windows CI coverage and exercise both Unix and TCP endpoints
on Linux.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Docker Linux, Debian Bookworm with procps and git, Python 3.12.12,
--init, a native Linux source copy, and a frozen environment: 783
related unit tests and all 3 real-server integration tests passed. The
source and test files were compared with the committed versions before
publication.
```sh
uv sync --frozen --extra dev
uv run --frozen --no-sync pytest tests/test_mcp_transport.py tests/test_mcp_bridge.py tests/test_mcp_endpoint.py tests/test_cli_mcp_commands.py tests/test_subagent_acp_mcp_lifecycle.py tests/test_node_client.py tests/test_acp_journal.py tests/test_subagent_acp_remote.py tests/test_subagent_manager.py tests/test_subagent_mcp_grant.py tests/test_subagent_third_party.py -q -n 4 --dist loadfile --tb=short --cov=raven --cov-report=json:/out/coverage.json --junitxml=/out/units.xml
uv run --frozen --no-sync pytest tests/integration/test_mcp_bridge_e2e.py -m integration -q -n 0 --tb=short --junitxml=/out/integration.xml
```
PR diff coverage is 98.26% (169/172 changed executable lines), above the
unchanged 90% threshold. On the host checkout, `$coverageReport` points
to the JSON exported by that Linux run:
```powershell
uv run --frozen --no-sync python scripts/coverage_gate.py --coverage-json $coverageReport diff --base-ref origin/main --head HEAD --threshold 90
```
Native Windows, Python 3.13.12, a separate frozen environment and
PYTHONUTF8=1: all 21 bridge unit tests passed; all 8 integration cases
passed, comprising 3 MCP bridge cases and 5 native command-line cases
retained in the Windows CI job.
```powershell
uv run --frozen --python 3.13 --no-sync pytest tests/test_mcp_bridge.py -q -n 0 --tb=short
uv run --frozen --python 3.13 --no-sync pytest tests/integration/test_mcp_bridge_e2e.py tests/integration/test_acp_mirasim_real_process.py -m integration -q -n 0 --tb=short
```
Seven regression cases cover paused input, resuming consumption, closing
a paused worker, a busy event loop, and teardown with pending TCP
writes: while the endpoint stays stalled, while it drains those writes,
and when a second cancel arrives after that drain finished. Aborting the
endpoint after a close that finished was wrong: the selector transport
has already run connection_lost and dropped its loop, and its
_force_close returns early only on _conn_lost, which a drained close
does not raise, so the bridge exited 1 at teardown on Linux and macOS.
The abort now depends on whether the endpoint's write buffer still holds
bytes, since a closing transport empties it before calling
connection_lost. A flag set once the close returned would miss the
second cancel, and that case fails with one. The stalled-endpoint
teardown case pauses the endpoint's reading before the frames are fed:
it used to wait for the socket to stop reading through its own flow
control, which took 1.87 s of the case's 2 s budget on Linux, so a busy
runner timed out before the teardown ran. It is now 0.17 s per case, and
20 consecutive runs under 32 busy loops on 16 cores all passed. The
original teardown and buffering cases fail without the fixes. At this
head, making the stdin worker read again without waiting for its
previous chunk to be delivered fails 3 cases, removing the close
deadline fails 2, and never aborting fails 2; aborting unconditionally
fails both drained cases. These mutations ran on a disposable copy of
the source tree, not on the committed files.
The unreachable-upstream integration test still uses its original
1.5-second deadline. Its Windows fixture bypasses the venv redirector
while preserving the venv, so an inherited stdout handle cannot mask
EOF.
Full-PR pre-commit, commitlint, commit-message, source-language, asset
and whitespace checks passed. The bridge type check passed and all 10
import contracts were kept:
```powershell
uv run --frozen --no-sync pre-commit run --from-ref origin/main --to-ref HEAD
npx --no-install commitlint --from origin/main --to HEAD --config commitlint.config.cjs
uv run --frozen --no-sync python scripts/check_commit_messages.py origin/main..HEAD
uv run --frozen --no-sync python scripts/check_large_files.py origin/main...HEAD
uv run --frozen --no-sync python scripts/check_source_language.py origin/main...HEAD
uv run --frozen --no-sync ty check raven/mcp/bridge.py
uv run --frozen --no-sync lint-imports
git diff --check origin/main...HEAD
```
These are focused local results. Remote dispatch tests use the
repository's SSH stand-in and real node processes, not an external
machine. POSIX-dependent paths are covered on Linux; no full-suite
Windows pass, live LLM run or sandbox VM run is claimed. Existing
Windows path, shell and journal portability outside this MCP change
remains out of scope.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
Windows dispatches can now use granted MCP servers. Each fallback
endpoint opens an ephemeral loopback listener with a bounded
authentication handshake and its own random token. Upstream definitions
and credentials remain on the host, and remote dispatches receive no
loopback token.
Input now pauses when a bridge cannot forward requests. During shutdown,
a connection that cannot flush within one second is aborted and its
remaining queued bytes can be discarded. This bounds cleanup when a peer
has stopped reading.
Unix sockets remain the preferred transport, and the Raven node wire
protocol is unchanged. No dependencies, configuration fields or
migrations are added. Reverting the change restores the previous Windows
MCP limitation.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Fixes #962
---------
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-19 19:58:55 +08:00
|
|
|
endpoints = McpEndpoints()
|
|
|
|
|
cfg = MCPServerConfig(
|
|
|
|
|
command=sys.executable, args=["-c", "import sys; [print(line, end='', flush=True) for line in sys.stdin]"]
|
|
|
|
|
)
|
|
|
|
|
path = await endpoints.open("d1", "pg", cfg)
|
|
|
|
|
try:
|
|
|
|
|
pool.lend("box", "d1", {"pg": str(path)}, task="agent")
|
|
|
|
|
await pool._on_stream("box", protocol.MCP_OPEN, {"stream": 1, "task": "agent", "lend": "d1", "server": "pg"})
|
|
|
|
|
frame = '{"jsonrpc":"2.0","id":1,"method":"ping"}'
|
|
|
|
|
await pool._on_stream("box", protocol.MCP_FRAME, {"stream": 1, "text": frame})
|
|
|
|
|
await _until(lambda: bool(_told(client, protocol.MCP_FRAME)))
|
|
|
|
|
assert json.loads(_told(client, protocol.MCP_FRAME)[0]["text"])["method"] == "ping"
|
|
|
|
|
await pool.unlend("box", "d1")
|
|
|
|
|
assert not pool._nodes["box"].streams
|
|
|
|
|
assert _told(client, protocol.MCP_CLOSE)[-1]["reason"] == "the dispatch ended"
|
|
|
|
|
finally:
|
|
|
|
|
await pool.close_all()
|
|
|
|
|
await endpoints.aclose()
|
feat(*): lend the host's MCP servers to an acp agent on a machine through its node (#947)
## Summary
Adds MCP lending for remote ACP agents on top of the machine and
recovery
services already merged in #920, #922 and #934. The branch includes
current
main and its default-on registered-machine behavior.
What this does: an acp entry on a registered machine is handed the MCP
servers its row grants, the way an entry on this computer is, and the
servers themselves stay here with their definitions and keys.
- The agent on the machine runs the node's own bridge as a stdio MCP
server (`python -I -m raven.node --mcp --dir <node dir> --task <task>
--lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte
pumps around a connection to the daemon's socket, no MCP parsed, no
credential held, the standard library only.
- The daemon carries the bridge's frames as a *stream* to the host
holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6,
`raven/node/protocol.py`, `raven/node/server.py`). The daemon tells
which
task a bridge is by the process that connected -- the peer pid off the
socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the
parent chain to the agent the daemon started, which the bridge is under
and no other task's process is; the kernel supplies that pid and a peer
cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the
task it names must be the one the connection is a process of. Two tasks
on
one machine run as one user, so no name or file a bridge could present
would tell them apart; the connecting process does. One stream per
bridge, at most 16 per task; a frame
travels in pieces of at most 256 KiB characters, the last marked
(`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the
daemon relays and keeps nothing. A stream ends with either end, the
host's link or the task, and the other end is told.
- The host opens the endpoints for a machine row exactly as for a row
here
(`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in
the host's sandbox, HTTP auth added here), binds them to the machine,
dispatch and acquired agent task
(`NodePool.lend`/`unlend`) before handing out bridge stanzas. On
`mcp/open`
the host checks the daemon-authenticated task against that binding
before
connecting; it rechecks the grant and link after the socket opens. An
unlend, replacement grant or replaced link cannot authorize that pending
connection. The host then pumps the frames both ways
(`raven/node/client.py`).
`node/hello` names the bridge; a node serving without a daemon
(`--stdio`)
names none, and the dispatch then goes without MCP and says so.
- The reply closes with the servers lent and the machine's node, or with
why none were. Remote lending requires Unix-socket support on the host:
on Windows it is withheld before endpoint setup, the task continues, and
the reply explains why. The roster's "injectable MCP" matches that
support
(`session_mcp_delivered`). The cli lane on a machine still
withholds its servers (the next change); the openai lane has no tools.
Not in this change: the cli lane's MCP file on the machine; a stream
that
survives the link going (a bridge sees its server end, the agent reports
the call failed, and the next dispatch works again); MCP for a run
picked
up after this computer restarted (#934): it reports its calls fail.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Follow-up fix 05eaba67a, after the c9de53947 review:
- The nine tests the Windows review named now pin the host fact through
one
fixture, `unix_socket_host` (`tests/conftest.py`), which sets
`raven.mcp.bridge.unix_sockets_available` to true and re-points every
`raven.*` module that imported it by name; their bodies are unchanged.
- Found complete, not just the nine: the whole suite was run with
`asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted
before collection (a pytest plugin, controller and xdist workers), at
the
merge base `6b938a426` and at `c9de53947`, and the two failure sets
compared. 17 node ids failed at the head and not at the base: the nine,
and eight this change added that run the transport for real (the bridge
module, and two lending tests through the stand-in machine). Those eight
are guarded by host capability instead of pinned: the bridge module
skips
when `sys.platform == "win32" or not unix_sockets_available()`, the two
lending tests carry the existing `skipif(not unix_sockets_available())`.
After the fix the same run fails the same 66 node ids at the head as at
the base: nothing this change adds or touches fails on a host without
unix sockets.
- A bridge whose daemon side goes ends as streams end. The host can
close a
stream while a frame of the agent's is on its way; the daemon writes
`mcp/close` and hangs up, and the frame meets the closed socket, or the
read side is reset first. That exited 1 with `BrokenPipeError` or
`ConnectionResetError`; a connection-level end is now exit 0 with
nothing
said, a frame past the cap still exits 1 with its reason, both pumps are
awaited before the socket closes, and a daemon gone before the stream
opens is one sentence, not a traceback. Four deterministic cases with a
stand-in connection fail on `c9de53947` and pass here.
- The bridge test's stand-in agent outlives its bridge, as an agent
outlives
one of its MCP servers, so the later parts of
`test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins`
no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80
here.
- The focused set (node, MCP, remote, manager and roster files) -- 935
passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
simulation cases, which fail the same way on main on this computer);
diff coverage against main 97.67% (420/430). Real machine, node
reinstalled with this tree in a directory of the check's own: a stdlib
ACP
agent's bridge reached a host-held server and 1 MiB and 8 MiB results
crossed whole, bridge exit 0; nothing was left on the machine.
Follow-up fix c9de53947, after the same-user token finding:
- The daemon no longer uses a token. `agent/launch` mints and writes
none,
`mcp.token` and the `hmac` check are gone, and `Daemon.open_stream`
identifies the bridge by the connecting process instead (`_peer_pid` +
`Daemon.task_of`). The same-user path the reviewer kept open -- a peer
reading another task's `mcp.token` and opening its stream -- is closed,
because identity is the connection the kernel reports, not a readable
secret.
- Tests, each failing on the unfixed tree first: a bridge placed under
task
B that names task A is refused (`the connection is a process of task
'agent@box-2', not 'agent@box-1'`), as are a process under no task, an
unplaced connection, and a task the daemon does not run
(`tests/test_node_server.py`); a real-OS test launches an agent, takes
its
real child, and asserts the child resolves to the task while the test's
own
process resolves to none (runs on Linux in CI); `_peer_pid` is checked
against a real socketpair; the in-process and subprocess bridge tests
now
identify by the connecting process, and a bridge this test spawns
itself,
or another task's agent spawns, never opens the stream.
- Windows withholding now covers a row on this computer too, not only a
machine row: `session_mcp_delivered` drops the `machine and` guard and
answers no whenever `unix_sockets_available()` is false, so a local acp
row is no longer advertised as injectable and no longer raises at
dispatch
on a host without the transport; `_mcp_endpoints` yields nothing before
opening an endpoint and `_mcp_note` explains it. The two tests the
Windows
review named now guard on `unix_sockets_available()`, and the
withholding
test is parameterized over a machine row and a local row.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over the node/MCP/remote suite -- 97.63% changed-line coverage
(412/422).
`ruff`, `ty` on the changed modules, and the import-cycle budget pass.
- Real machine (ravenx2, this head, node reinstalled, digest
fdbe132e76f0):
a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB
and
an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the
connecting-process identity admits the legitimate bridge end to end on
real Linux; the cross-task refusal on Linux is the real-OS test above,
which CI runs there.
Earlier review fixes (4dc5b3e1f, the task-to-grant binding). Its
host-side
binding of each dispatch's endpoints to the acquired agent task is kept;
the
daemon-side token this run verified was since replaced by the
connecting-process
identity above (c9de53947), so the token-named tests and reproduction
below no
longer describe the current tree.
- `uv run --no-sync pytest tests/test_node_client.py
tests/test_node_server.py tests/test_node_bridge.py
tests/test_node_relay.py tests/test_node_bundle.py
tests/test_node_install.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py
tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict
--cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report=
--tb=short` -- 1495 passed and one minimal-node stdio response test
failed;
strict idle accounting also flagged the assembly test. Both cases pass
together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or
timing limits were weakened. The new authorization, revocation and host
compatibility tests all pass.
- The five focused cases failed on the unfixed implementation first: a
different authenticated task selecting another grant, a stream with no
task, each missing Unix socket API, and an astral Unicode chunk over the
node's byte limit. Regressions also cover revocation, grant replacement
and link replacement while an endpoint connection is opening, and a real
remote turn continuing when both host Unix socket APIs are absent.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 97.94% (381/389 changed executable lines); this fix alone is 100%
(40/40). Python lint, type, dependency
and import checks pass; all 10 import contracts hold. The scope checks
for commit messages, source language and file sizes pass. Both RPC
generators pass `--check` after integrating current main.
- A real Linux node with two concurrent tasks and a host loopback
endpoint:
before, the attacker's own task token plus the victim's public grant
names returns the victim endpoint's marker; after, it returns no data
and the endpoint is never reached. Its own grant works, unknown and
revoked grants fail, and the same attack remains denied after the relay
is killed and an ordinary ping reattaches both tasks. A full chunk of
262144 astral characters was lost/corrupted before and arrives intact
after. The final documented source tree was reinstalled and checked
again.
- Real ACP dispatch through the node's bridge reaches a host-held stdio
MCP tool and returns its marker; its grant is removed when the turn
ends.
With the host Unix socket APIs removed to simulate Windows, the remote
agent still completes, receives no MCP, explains the host limitation,
and the host MCP ledger records no extra call. This is not a native
Windows run. Dedicated test tasks, host endpoints and bridge processes
are cleaned up.
- CONTEXT.md, module/field docstrings, both collaboration guides, the
external design notes and this description were checked against the
implementation. Main's no-debug-switch behavior and full-turn recovery
documentation are retained.
Full combined-tree verification: `uv run --no-sync pytest -n 4
--tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure
reproduces on current main in a separate checkout: host CLI color
rendering,
LibreOffice detection and the two simulation cases. No other test fails.
The minimal-node response test that failed in the focused parallel
selection
passes in this full run. Diff coverage: 97.94% (381/389 executable
changed lines).
Earlier author verification:
- `uv run pytest tests/test_node_bridge.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_bundle.py tests/test_node_install.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py`
-- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge
and a stdio node naming none, the token written owner-only beside the
record, a stream opened with the task's token and refused with another
or for a task not running, refused with no host holding the task,
refused past 16 per task, one per bridge connection, the pieces of a
frame crossing both ways as they are and a stranger carrying nothing,
and every way a stream ends (the bridge closing, the host closing, the
bridge's connection ending, the host's link ending, the agent ending)
with the other end told; the bridge as a process against a real daemon
over its socket: a frame to the host and one back whole, a large frame
each way in pieces, the agent closing its stdin ending the stream and
the bridge leaving clean, the host closing the stream ending the bridge,
a wrong token and a missing daemon each refused on stderr with exit 1;
the stand-in machine with the real daemon and the real bridge: an acp
stub there that connects to its `mcpServers` and calls a tool reaches a
stdio server here and answers with its result, the stanza handed to it
naming the node's bridge, the task, the dispatch and the server and
nothing of the server's definition or socket, the ledger here showing
one call, no bridge outliving the dispatch; the reply's note and the
roster predicate for a machine row.
- `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed,
4 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,
`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`,
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`
and
`test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`,
which fail
the same way on main on this machine.
- Real machine (a registered Linux machine with Claude Code signed in,
the node reinstalled with the code before and after):
A28, before (the
code of #934's head and its node): the turn closed with "MCP servers
'probe' were not lent", Claude Code said no tool of that name was
connected, and the server here saw no call; after: `PROBE_OK:r2-lend`
in 9.4 s, the server here recorded the call, and the turn closed with
"lent to acp agent ... through the Raven node on machine ...". A34:
after
the turn the machine's process list shows no bridge, a grep of its node
directory finds nothing of the server's command or of this computer's
socket directory, and no endpoint socket is left here. A29: with the
tool switched off here, Claude Code answered UNAVAILABLE and the server
saw no call. A30: a 40 s tool call under way, the bridge visible in the
machine's process list, then cancelled: 3 s later no bridge there and
no socket here. A31: the relay ssh killed during a 20 s call: the turn
ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next
dispatch reached the server again in 7.6 s. A32: five calls in one turn
all reached the server and came back; an 8 MiB answer: the server here
produced it, and a plain ACP agent on the machine (a stdlib stub that
speaks the stanza's bridge directly) received it whole through the node
-- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude
Code itself closed the server on that answer and reported the call
failed: the agent's own cap on a tool result, not the link's.
- `ruff`, the commit-message and PR-body checks -- clean.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
The English and Chinese collaboration guides describe lending,
task-bound authorization,
Windows withholding, and the existing CLI/restart limitations.
## Risk
Registered-machine support is enabled by configuration, without the
removed
trial environment switch. An ACP entry on a machine receives its granted
MCP servers on a Unix-socket-capable host; the host's normal MCP policy
still
applies. This intentionally exposes those tools to the selected remote
agent.
The server definition, credentials and endpoint paths remain on the
host.
The daemon identifies a bridge's task by the connecting process (the
peer
pid the kernel reports, walked to the agent it started), not by any
token or
name the bridge presents, so a peer task sharing the OS account cannot
open
another task's stream; the host then authorizes that task against the
dispatch/server grant. A system that reports no peer pid opens no
stream.
On Windows hosts remote lending is withheld explicitly, without aborting
the
remote task. Protocol 6 remains the MCP wire version; host and node must
also
have the same code digest, so updated nodes must be reinstalled.
Rollback is
a revert; the machine then carries no stream and receives no bridge
stanza.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
#934 (merged); #920 and #922 (merged).
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-15 15:37:50 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_lending_requires_a_nonempty_task():
|
|
|
|
|
pool = NodePool()
|
|
|
|
|
with pytest.raises(ValueError, match="agent task"):
|
|
|
|
|
pool.lend("box", "d1", {"pg": "/unused"}, task="")
|
|
|
|
|
assert pool._lent == {}
|
feat(*): a machines hub tab, and registering a machine with a password Raven keeps (#1005)
## Summary
Add a Machines tab to the Agent Hub and let owners register an SSH
machine with either a key or a password.
- The Machines tab reuses the existing machine cards, details and
registration form. The hub remembers the last selected module. Machine
details show the login method, and removal explains that the locally
stored password is deleted.
- Password registrations keep a 0600 credential file under the Raven
home's 0700 keys directory. Registration, node installation, file tools,
tunnels and Oncall use the shared SSH askpass transport. Passwords are
excluded from command lines, registry rows and responses, and included
in held-secret scrubbing.
- Password connections authenticate independently, disable inherited
jump hosts, and answer only the registered account's password prompts.
Oncall cannot redirect a stored password to another endpoint. Windows
rejects this mode before creating credentials because its OpenSSH cannot
execute Raven's POSIX helper. Key authentication remains supported as
before.
- Concurrent registrations preserve the successful request's password.
Each probe uses its own temporary 0600 credential file, which is
scrubbed and removed afterwards. Password writes receive a file
identity; cleanup checks ownership under the writer's lock. Successful
registration checks duplicates again and publishes its password with the
registry row under that lock. Both completion orders, RPC cancellation
and CLI aborts are covered.
The RPC contract adds optional `MachineRow.auth` and password
registration parameters. English and Chinese machine documentation, the
runtime context and changelog describe the resulting behavior.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
After rebasing onto the latest main:
```bash
uv run --no-sync pytest tests/test_ops_connection_add.py tests/test_ops_connections.py tests/test_ops_transport.py tests/test_cli_ops_connection_commands.py tests/test_rpc_machines.py tests/test_held_secrets_machine_passwords.py tests/test_acp_remote.py tests/test_node_client.py tests/test_rpc_schema_match.py tests/test_rpc_registration.py tests/test_import_cycle_budget.py -q -n 0 --tb=short --cov=raven.ops.connection_add --cov=raven.ops.transport --cov=raven.cli.ops_connection_commands --cov=raven.rpc.methods.machines --cov-report=json:/tmp/pr1005-fix-coverage.json
```
Result: 932 passed, 6 skipped. The skipped cases require zsh, tcsh or
csh, which are absent on this Linux host.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
/tmp/pr1005-fix-coverage.json diff --base-ref HEAD^ --threshold 99`:
100% (94/94 executable changed lines) for the concurrency fix.
- `uv run --no-sync pre-commit run --from-ref HEAD^ --to-ref HEAD`:
passed, including formatting, file checks and private-key detection.
- `uv run --no-sync ty check raven/ops/connection_add.py
raven/ops/transport.py raven/cli/ops_connection_commands.py
raven/rpc/methods/machines.py`: passed.
- `uv run --no-sync lint-imports`: all 10 contracts passed.
- `uv run --no-sync python scripts/check_source_language.py
origin/main..HEAD`, `uv run --no-sync python
scripts/check_commit_messages.py origin/main..HEAD`, `make
check-large-files`, and `git diff --check origin/main...HEAD`: passed.
The original RPC failure/cancellation and CLI-abort reproductions failed
before the fix. Additional regressions cover the reverse completion
order, probes with competing passwords, temporary-secret scrubbing,
duplicate registration and failed registry writes. The complete rebased
source was compared with the original PR plus main, and the concurrency
fix retained its patch identity.
## Risk
- [x] Security impact considered: stored and temporary passwords are
owner-readable and scrubbed from tool output. Unknown SSH host keys are
accepted on the first connection, as before; the guides explain
verifying the host from a terminal first on an untrusted network.
Passwords shorter than six characters or containing CR/LF are rejected.
- [x] Backward compatibility considered: key registration remains the
default and `MachineRow.auth` is optional. Password publication and
cleanup now use file locks and atomic replacement; temporary probe files
are removed when the probe ends.
- [x] Rollback path is clear: revert this PR to restore the earlier
registration flow. Removing a machine deletes the password stored under
its id. Existing password rows require migration to key authentication
if reverting the feature.
## Related Issues
N/A
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn>
Co-authored-by: MiniMax (minimax-m3.1-flash-free) <noreply@minimax.io>
Co-authored-by: Codex <noreply@openai.com>
2026-10-20 12:23:40 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
# ---- a machine registered with a password: the link logs in through askpass ------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _password_row() -> dict[str, Any]:
|
|
|
|
|
return {k: v for k, v in row().items() if k != "key"} | {"auth": "password"}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.slow
|
|
|
|
|
async def test_the_link_to_a_password_machine_logs_in_through_askpass_and_offers_no_key(
|
|
|
|
|
box, pool, monkeypatch, supported_host
|
|
|
|
|
):
|
|
|
|
|
from raven.ops.connection_add import store_password
|
|
|
|
|
|
|
|
|
|
monkeypatch.setenv("RAVEN_HOME", str(box.root / "rhome"))
|
|
|
|
|
box.rows([_password_row()])
|
|
|
|
|
box.password("hunter2")
|
|
|
|
|
store_password("box", "hunter2")
|
|
|
|
|
box.node()
|
|
|
|
|
|
|
|
|
|
hello = await pool.hello("box")
|
|
|
|
|
|
|
|
|
|
assert hello["digest"] == bundle.digest()
|
|
|
|
|
(argv,) = box.launches()
|
|
|
|
|
assert "-i" not in argv and "PubkeyAuthentication=no" in argv and "hunter2" not in " ".join(argv)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_password_the_machine_refuses_ends_the_link_with_the_login_named_and_nothing_else(
|
|
|
|
|
box, pool, monkeypatch, supported_host
|
|
|
|
|
):
|
|
|
|
|
from raven.ops.connection_add import store_password
|
|
|
|
|
|
|
|
|
|
monkeypatch.setenv("RAVEN_HOME", str(box.root / "rhome"))
|
|
|
|
|
box.rows([_password_row()])
|
|
|
|
|
box.password("hunter2")
|
|
|
|
|
store_password("box", "wrong-one")
|
|
|
|
|
box.node()
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError) as caught:
|
|
|
|
|
await pool.hello("box")
|
|
|
|
|
|
|
|
|
|
said = str(caught.value)
|
|
|
|
|
assert "refused the login" in said and "password" in said
|
|
|
|
|
assert HOST not in said and "wrong-one" not in said
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
async def test_a_password_machine_whose_password_is_not_kept_here_is_refused_before_ssh_is_started(
|
|
|
|
|
box, pool, monkeypatch, supported_host
|
|
|
|
|
):
|
|
|
|
|
monkeypatch.setenv("RAVEN_HOME", str(box.root / "rhome"))
|
|
|
|
|
box.rows([_password_row()])
|
|
|
|
|
box.node()
|
|
|
|
|
|
|
|
|
|
with pytest.raises(NodeError, match="no password is kept here for machine 'box'"):
|
|
|
|
|
await pool.hello("box")
|
|
|
|
|
|
|
|
|
|
assert box.launches() == [], "ssh never ran"
|