Raven-1/tests/test_node_install.py

1086 lines
46 KiB
Python
Raw Permalink Normal View History

feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
"""raven.node.install: putting this Raven's node on a registered machine.
The steps are shell text run on the machine, so the parts that decide
something -- the probe, the prune -- run here under a real ``sh`` against a
stand-in tree, and one install runs for real on this computer: a virtualenv of
this interpreter, this Raven's code unpacked into it, and the node that makes
reporting this Raven's digest. Only the package index is stood in for; no test
reaches the network.
Every failure is one sentence that names the machine, never its address, and
a step that fails leaves no half-made node behind.
"""
from __future__ import annotations
import dataclasses
import json
import shutil
import stat
import subprocess
import sys
from pathlib import Path
from typing import Any
import pytest
from raven.acp_client import remote
from raven.node import bundle, install, protocol
from raven.node.install import InstallError, Plan
HOST = "203.0.113.7"
PORT = 58717
ROW = {"id": "box", "display_name": "Lab box", "host": HOST, "port": PORT, "user": "worker", "key": "/keys/id_test"}
@pytest.fixture
def registry(tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
path = tmp_path / "connections.json"
def write(**extra: Any) -> None:
path.write_text(json.dumps({"connections": [{**ROW, **extra}]}), encoding="utf-8")
write()
monkeypatch.setenv("RAVEN_CONNECTIONS", str(path))
return write
def _target(**extra: Any) -> remote.Machine:
return remote.Machine(id="box", display_name="Lab box", _row={**ROW, **extra})
def _plan(**fields: Any) -> Plan:
base: dict[str, Any] = {
"target": _target(),
"node_dir": "~/.raven-node",
"install_name": bundle.install_name(),
"present": False,
"python": "/usr/bin/python3.12",
"uv": "/usr/bin/uv",
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
# What a probe of that uv reports: one recent enough to read a configured index.
"uv_version": install.UV_VERSION,
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
"free_kb": 10 * 1024 * 1024,
"fetch": "/usr/bin/curl",
}
return Plan(**{**base, **fields})
class Machine:
"""Answers each step by name, records what it was sent, and fails where told."""
def __init__(self, **answers: tuple[int, str]) -> None:
self.answers = answers
self.sent: list[str] = []
self.pushed: list[bytes] = []
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
self.pushed_by: list[tuple[str, bytes]] = []
"""Each push with the command it fed: what reached a step's stdin."""
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
self.check = json.dumps({"protocol": protocol.PROTOCOL, "digest": bundle.digest()})
def _answer(self, command: str) -> tuple[int, str]:
for name, cmd in install.steps(_plan()) + install.steps(_plan(python="", uv="/usr/bin/uv")):
if command == cmd and name in self.answers:
return self.answers[name]
if command.endswith("-I -m raven.node --version"):
return 0, self.check
if " du -sk " in f" {command}":
return self.answers.get("du", (0, "51234\n"))
if "echo RG=1 || echo RG=0" in command:
return 0, "RG=1\n"
return 0, ""
def run(self, target: remote.Machine, command: str) -> tuple[int, str]:
assert target.id == "box"
self.sent.append(command)
return self._answer(command)
def push(self, target: remote.Machine, command: str, data: bytes) -> tuple[int, str]:
self.sent.append(command)
self.pushed.append(data)
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
self.pushed_by.append((command, data))
if command.endswith('tar -xzf - -C "$node/lib"'):
return self.answers.get("code", (0, ""))
return self._answer(command)
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
@pytest.fixture
def small_tarball(monkeypatch: pytest.MonkeyPatch) -> bytes:
monkeypatch.setattr(bundle, "tarball", lambda root=None: b"TAR")
return b"TAR"
def test_each_step_runs_through_the_registry_s_ssh_with_the_step_budget(monkeypatch):
seen: list[tuple] = []
monkeypatch.setattr(
remote, "run", lambda target, command, *, timeout: seen.append(("run", command, timeout)) or (0, "")
)
monkeypatch.setattr(
remote,
"push",
lambda target, command, data, *, timeout: seen.append(("push", command, data, timeout)) or (0, ""),
)
assert install._run(_target(), "true") == (0, "")
assert install._push(_target(), "cat", b"x") == (0, "")
assert seen == [("run", "true", 900.0), ("push", "cat", b"x", 900.0)]
# --- one look at the machine ------------------------------------------------------
def test_the_probe_s_answers_become_the_plan(registry):
sent: list[str] = []
def run(target: remote.Machine, command: str) -> tuple[int, str]:
sent.append(command)
return (
0,
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
"bash: no job control\nPRESENT=1\nPYTHON=/usr/bin/python3.12\nUV=/root/.local/bin/uv\nUV_VERSION=0.13.0\n"
"FREE_KB=2097152\n",
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
)
p = install.plan("box", run=run)
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
assert (p.node_dir, p.install_name, p.present, p.python, p.uv, p.uv_version, p.free_kb) == (
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
"~/.raven-node",
bundle.install_name(),
True,
"/usr/bin/python3.12",
"/root/.local/bin/uv",
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
"0.13.0",
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
2097152,
)
assert p.target.label == "'Lab box' (box)"
(script,) = sent
assert bundle.digest() in script and HOST not in script
def test_a_machine_with_nothing_found_plans_nothing_present(registry):
p = install.plan("box", run=lambda target, command: (0, "FREE_KB=\n"))
assert (p.present, p.python, p.uv, p.free_kb, p.how) == (False, "", "", None, "")
@pytest.mark.parametrize(
("registered", "asked", "want"),
[
(None, None, "~/.raven-node"),
("/data/raven-node/", None, "/data/raven-node"),
("/data/raven-node", " /tmp/nodes/ ", "/tmp/nodes"),
],
)
def test_the_node_dir_is_the_one_asked_for_else_the_registry_s(registry, registered, asked, want):
if registered is not None:
registry(node_dir=registered)
p = install.plan("box", node_dir=asked, run=lambda target, command: (0, ""))
assert p.node_dir == want
def test_a_machine_that_cannot_be_looked_at_is_named_without_its_address(registry):
def run(target: remote.Machine, command: str) -> tuple[int, str]:
return 255, f"ssh: connect to host {HOST} port {PORT}: Connection refused"
with pytest.raises(InstallError) as caught:
install.plan("box", run=run)
assert str(caught.value) == "machine 'Lab box' (box) could not be reached"
def test_a_look_that_failed_past_ssh_says_so(registry):
with pytest.raises(InstallError, match=r"^could not look at machine 'Lab box' \(box\)$"):
install.plan("box", run=lambda target, command: (2, "df: weird"))
feat(*): pick up a turn left on a registered machine when this computer stops (#934) ## Summary Registered machines are now available without a debug environment switch. `RAVEN_DEBUG_MACHINES` previously defaulted to off: it kept machine-backed agents out of the dispatch table, refused node/installer access, hid remote preset destinations, and treated file paths locally while off. It was a trial feature gate, not a verbose-logging option. At the owner's request, this PR removes the variable and all of those checks. Normal startup and restart recovery use the configured machine entries; even a leftover value of `0` is ignored. Machine registration, SSH authentication, file permission prompts, and concurrency limits still apply. Fourth change in the series that lets a Raven dispatch work to the machines in its machine registry. #920, #921 (merged through #922) and #922 are on main; this change stands on main alone now, and the diff is its own. What this does: a turn running on a registered machine survives this computer stopping, and the next start picks it up and reports it. - While a turn runs on a machine, this computer remembers what the machine does not know (`raven/acp_client/remote_runs.py`): one owner-only file per run under `<raven home>/machines/runs/`, with the machine's registry id, the task id its daemon runs the agent under, the agent, the session key, the instance handle, the dispatch id, the ACP session, the id of the prompt whose answer is awaited, the ids this side has used, how many of the agent's frames were read here, the agent's answer to `initialize` and the connection's launch key. Kept current after every frame the connection has dealt with, removed when the turn ends here. No address is in it. - When this process stops -- the gateway, `serve`, the TUI, the page stack -- the agents on machines stay with their daemons (`remote.leave_agents_running()` before the drain): a cancelled turn on a machine sends no `session/cancel`, closing its connection ends the endpoint only, and the manager leaves the dispatch's record open rather than finishing it as cancelled. A hard kill or `/restart` leaves the same record, written as the turn ran. - At the next start -- the gateway once its generation is bound; the `serve`, TUI and page stacks once -- `raven/agent/subagent/resume_remote.py` picks every run up: `task/list` on the machine, `agent/attach` from the frame last read (`NodePool.attach`), the connection adopted into the pool with request ids carried on past the other process's (`AcpConnectionPool.adopt`, `AcpClient.continue_ids_from`), the prompt's answer awaited (`AcpClient.await_response`), the reply built as a turn ended here is -- the partial notice when the agent stopped early, the machine note saying it was picked up after this computer restarted -- then the dispatch's record finished, the registry row and the node claim closed, and the result announced into the conversation that asked, as the manager announces a turn that ended here (`raven/acp_client/remote_resume.py`). A machine that cannot be asked keeps its record for a later start; a run the machine no longer has, one whose agent ended while this computer was away, one whose agent left the agent table, and one whose turn says nothing are announced as errors that say so. The adopted agent process serves later dispatches of its entry, as one started here would. - Seams: `AcpClient.request(sent=)` hears the id of what goes out; `ChannelEndpoint.watchers` hear the frames the connection has dealt with (`consumed`, called by the read loop); `ChannelEndpoint.stays` reads the leaving flag. - A cli entry's run on a machine is recorded while it runs (kind `cli`, no ACP part) and left the same way, the manager reading the dispatch as left; the entry picks it up (`CliAgentBackend.pick_up`): the daemon replays the command's whole output from its first line, the exit code ends it, and the reply closes with the machine note. - The record carries the task as the model wrote it, and -- once the manager has read the run as left -- the dispatch's origin and summary (`remote_runs.annotate`): the announce after a restart names the instance, node and tool call as the announce here does and quotes the task as authored, never the rendered prompt on disk; a host killed outright gets an origin made up from the spawn record and the run. - The record goes last, once the outcome has reached its conversation: a conclusion that fails, a stop under it, or an announce the scheduler refuses (`_announce_result` says so now) leaves the record, its claim released, for the next start to try again. - A run that returned here keeps its record the same way, until the manager's announce of the reply was taken: the manager dispatches with `reported_by_caller=True`, the backend then leaves the record behind with the reply instead of removing it as it returns, and the manager removes it after the announce (`remote_runs.forget_dispatch`) -- or, when the draining scheduler refused the announce, annotates it with the dispatch's origin and summary for the next start to report. A host killed between the agent's answer and the announce therefore still has the way back to the run. A caller that takes the reply as its own result (a graph node, a probe, a direct chat) dispatches without the keyword, and the record goes as it did. The record's frame count stops at the prompt: the turn's own frames, the chunks before the answer as much as the answer, are never checkpointed, so a pick-up replays the turn whole and the reply it reports is complete; a record that had moved past the answer would have resumed after it and waited forever. Not in this change: a run from the DAG lane has no spawn record to finish; it is announced from the origin the manager added to its record when it was left, or, for a host killed outright, from the run's record alone. The Tasks page reads the finished record on its next load rather than live. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Machine availability and restart checks after removing the trial switch: - New default-behavior regressions were run before the removal: 8 failed, with the enabled-value control passing. After removal, all 9 passed, including ACP/CLI recovery through the real AgentRegistry with the variable absent or set to `0`. - `uv run --no-sync pytest tests/test_acp_remote.py tests/test_node_client.py tests/test_node_server.py tests/test_node_install.py tests/test_ops_remote_path.py tests/test_permissions_remote_read.py tests/test_remote_file_tools.py tests/test_deliver_files_tool.py tests/test_raven_config_tool.py tests/test_rpc_subagents.py tests/test_shell_machine_channel.py tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py tests/test_resume_remote.py tests/test_remote_resume.py tests/test_remote_runs.py tests/test_subagent_manager.py tests/test_cli_tui_commands.py tests/test_rpc_schema_match.py tests/test_config_schema.py -n 4 --idle-ceiling-strict --cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report= --tb=short` -- 1799 passed on the combined tree, including the latest published result-retention fix. An earlier run had a missing stdio response in the minimal-node dependency test and an idle-gate hit; both passed a focused recheck, and this final complete selection passed with the strict gate. - `uv run --no-sync python scripts/coverage_gate.py --coverage-json <coverage-json> diff --base-ref origin/main --threshold 90` -- 92.92% (656/706 changed executable lines) for this PR; the gate-removal commit alone covers 2/2 new executable lines (the remainder deletes gates and updates descriptions). - `UV_NO_SYNC=1 make lint-python lint-imports lint-deps lint-types`, the commit/source-language/large-file scope gates and `git diff --check` pass. Both RPC generators' `--check` runs pass; `npm test -- scripts/gates/fixture-shape.test.mjs` from `ui-web` passes all five tests. - Real Linux machine, isolated node installations, separate dispatch and restart processes: before removal, a task started with the trial switch on was reported as an absent agent and its recovery record deleted when restarted without it. After removal, both absent and old `0` settings recover the full CLI result and read its output file by address. The absent-variable case kills the relay before host shutdown. The combined final tree was reinstalled and both scenarios repeated successfully. - With the variable absent, 12 further timing/boundary probes pass: early CLI exit and blank lines, relay loss before heartbeat with normal-call recovery, tunnel loss, ACP/CLI/OpenAI cancellation during disconnection, 11 launches admitting 8 and refusing 3, occupied tunnel port, and an unwritable prompt file. Three attachment, POSIX-command and IPv6 checks also pass; Windows command parsing is simulated, not native Windows. No test task or tunnel remains; the dedicated node directories are removed. - CONTEXT.md, configuration and RPC descriptions, transport/path docstrings, both collaboration guides and the owner's design notes were checked against the code. The guides explain the former trial switch and normal startup without it. The existing SSH, permission and limit tests continue to exercise their checks with no test-only switch setup. - `env -u RAVEN_DEBUG_MACHINES uv run --no-sync pytest -n 4 --tb=short --cov=raven --cov-branch --cov-report=json:<full-coverage-json> --cov-report=` -- 31345 passed, 230 skipped, 2 xfailed, 5 failed. A separate current-main checkout reproduces four failures: CLI color rendering on this host, LibreOffice detection, and the two simulation cases. The remaining failure was the direct-chat test stub rejecting `reported_by_caller` from the latest result-retention change; the published wrapper fix now forwards only keywords declared by its fake backend. The full suite was not rerun for that test-only correction. Final direct-chat and manager recheck: 277 passed with strict idle accounting. Full-PR changed-line coverage from this run is 96.18% (679/706). Earlier verification: - `uv run pytest tests/test_remote_runs.py tests/test_remote_resume.py tests/test_resume_remote.py tests/test_acp_client_resume.py tests/test_node_client.py tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py tests/test_cli_tui_commands.py tests/test_subagent_manager.py tests/test_rpc_bootstrap.py tests/test_cli_gateway_commands.py tests/test_acp_client_launch.py tests/test_subagent_acp.py tests/test_scope_canon.py tests/test_l4_entrances.py` -- all passed (605 passed). Covered: the record written owner-only, kept current per frame and per prompt, removed when the turn ends here and kept when this process leaves; a dispatch left at shutdown neither finished nor announced by the manager; an endpoint ended while leaving leaves the agent with the daemon (real daemon); a task picked up from the frame last read, with the watchers hearing each frame; a pick-up the machine refuses or cannot carry; an answer awaited for a request another process sent, ids carried on, no `session/cancel` while leaving; the reply read off replayed frames with the other sessions' frames ignored, a turn cut short marked partial, an unreachable machine keeping the record, a run the machine no longer has, a prompt never sent, an agent that ended while this computer was away, an answer waited out, a turn that said nothing, a pick-up beside a live connection closing its own; the manager-side conclusion: record finished and announced, an error announced and the record failed, an unreachable machine keeping the record, an agent off the table, a failure inside one pick-up not stopping the next, a run with no conversation, the once-per-process scheduling; a cli run recorded while it runs and left at shutdown, picked up whole on the real daemon behind the stand-in ssh, its pick-up's errors said (a non-zero exit, a machine that no longer has it, a link down, this computer leaving), a cancel that is not a leave ending the command with no record left; the TUI's stop saying the agents stay before the cancel and its start scheduling the pick-up; the record annotated with the dispatch's origin and summary when the manager reads it as left, the announce quoting the task as authored and naming instance, node and tool call, with and without the manager's fields; the record outliving a refused announce and a failed conclusion, claim released; a cli record read without an ACP session, an older record read with defaults; the record moving with the frames before the prompt and not past it, a dispatch's record forgotten by the manager's id with its claim, the manager dispatching with `reported_by_caller` and forgetting the record after an announce that was taken and annotating it after one that was refused; on the real daemon behind the stand-in ssh, an acp reply and a cli output returned with the record kept and the host "stopped" before the announce, the next start reading the same reply again whole, and a dispatch without the keyword leaving no record. End to end on the real daemon behind the stand-in ssh: a turn dispatched, this process "stopped" the way the gateway stops, the agent still running on the daemon, then a fresh pick-up reading its answer, finishing the record and announcing it, with the agent's process the pool's connection again. - `uv run pytest` on this tree, at an earlier revision of this change -- 29270 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine; diff coverage of this change against its base: 97.45% (153 of 157 executable changed lines), and 94.67% for the four changes against main; `lint-imports` 10 kept, 0 broken; `ty` clean. - Real machine (a registered Linux machine with Claude Code signed in, node reinstalled with this tree): process A dispatched a 25 s task, saw the prompt out after 2.4 s and the record keep up with the frames (6, then 10), then stopped the way the gateway stops 5.4 s after the dispatch, leaving the record and the agent running there; process B, started fresh, found the record and the running task, picked it up in 2.4 s, announced the agent's reply to the conversation with the machine note, finished the record, removed the run's record, held the agent's process as the pool's connection, and closing the pool ended the agent on the machine. This revision, the cli lane with the code before and after, the node reinstalled for each: a cli entry `sh -c 'echo $$ > pid.txt; sleep 20; echo done; cat'` was dispatched and the host stopped the gateway's way 2.7 s later. Before: no record, the command left running on the machine, and the next start found nothing to pick up. After: the record kept with the task as authored and the manager's origin, the command running on, and the next start picked it up in 16.8 s (the sleep), announcing `done` and `hi` with the instance, node and tool call and the machine note. The same with the link killed before the stop: recorded, left, picked up. With the machine's daemon ended between the stop and the pick-up, it ended its groups on its way out, the task's record there said exit -9, and the pick-up announced that as the run's error in 0.1 s, leaving nothing on the machine. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint`, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed The English and Chinese agent-collaboration guides explain normal startup, registered-machine setup and recovery, and the removed trial switch. ## Risk Configured machine entries are available by default; entries without a `machine` remain local. A remote-looking file path is resolved against the registry even without an environment variable; use an explicit local path such as `./a:b` for a local filename containing a colon. A formerly hidden machine entry is now available for dispatch, and an old variable set to `0` no longer disables it. Stopping Raven no longer cancels a turn running on a machine; it goes on there until the agent finishes or is cancelled, and its answer arrives in the conversation after the next start, as a sub-agent result that triggers a turn. One record per run in the raven home, owner-only, removed when the run is concluded; a machine that cannot be reached keeps it, so a run is reported at most once and never lost quietly. The adopted connection answers requests with ids past the ones the stopped process used, so a replayed frame resolves nothing sent after the restart. Rollback is a plain revert; a record left behind is then an inert file. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues #920 and #922 (merged); #921 was carried by #922. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com> Co-authored-by: Codex <noreply@openai.com>
2026-10-14 21:34:19 +08:00
def test_an_install_plan_needs_no_debug_switch(registry, monkeypatch):
monkeypatch.delenv("RAVEN_DEBUG_MACHINES", raising=False)
asked = []
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
feat(*): pick up a turn left on a registered machine when this computer stops (#934) ## Summary Registered machines are now available without a debug environment switch. `RAVEN_DEBUG_MACHINES` previously defaulted to off: it kept machine-backed agents out of the dispatch table, refused node/installer access, hid remote preset destinations, and treated file paths locally while off. It was a trial feature gate, not a verbose-logging option. At the owner's request, this PR removes the variable and all of those checks. Normal startup and restart recovery use the configured machine entries; even a leftover value of `0` is ignored. Machine registration, SSH authentication, file permission prompts, and concurrency limits still apply. Fourth change in the series that lets a Raven dispatch work to the machines in its machine registry. #920, #921 (merged through #922) and #922 are on main; this change stands on main alone now, and the diff is its own. What this does: a turn running on a registered machine survives this computer stopping, and the next start picks it up and reports it. - While a turn runs on a machine, this computer remembers what the machine does not know (`raven/acp_client/remote_runs.py`): one owner-only file per run under `<raven home>/machines/runs/`, with the machine's registry id, the task id its daemon runs the agent under, the agent, the session key, the instance handle, the dispatch id, the ACP session, the id of the prompt whose answer is awaited, the ids this side has used, how many of the agent's frames were read here, the agent's answer to `initialize` and the connection's launch key. Kept current after every frame the connection has dealt with, removed when the turn ends here. No address is in it. - When this process stops -- the gateway, `serve`, the TUI, the page stack -- the agents on machines stay with their daemons (`remote.leave_agents_running()` before the drain): a cancelled turn on a machine sends no `session/cancel`, closing its connection ends the endpoint only, and the manager leaves the dispatch's record open rather than finishing it as cancelled. A hard kill or `/restart` leaves the same record, written as the turn ran. - At the next start -- the gateway once its generation is bound; the `serve`, TUI and page stacks once -- `raven/agent/subagent/resume_remote.py` picks every run up: `task/list` on the machine, `agent/attach` from the frame last read (`NodePool.attach`), the connection adopted into the pool with request ids carried on past the other process's (`AcpConnectionPool.adopt`, `AcpClient.continue_ids_from`), the prompt's answer awaited (`AcpClient.await_response`), the reply built as a turn ended here is -- the partial notice when the agent stopped early, the machine note saying it was picked up after this computer restarted -- then the dispatch's record finished, the registry row and the node claim closed, and the result announced into the conversation that asked, as the manager announces a turn that ended here (`raven/acp_client/remote_resume.py`). A machine that cannot be asked keeps its record for a later start; a run the machine no longer has, one whose agent ended while this computer was away, one whose agent left the agent table, and one whose turn says nothing are announced as errors that say so. The adopted agent process serves later dispatches of its entry, as one started here would. - Seams: `AcpClient.request(sent=)` hears the id of what goes out; `ChannelEndpoint.watchers` hear the frames the connection has dealt with (`consumed`, called by the read loop); `ChannelEndpoint.stays` reads the leaving flag. - A cli entry's run on a machine is recorded while it runs (kind `cli`, no ACP part) and left the same way, the manager reading the dispatch as left; the entry picks it up (`CliAgentBackend.pick_up`): the daemon replays the command's whole output from its first line, the exit code ends it, and the reply closes with the machine note. - The record carries the task as the model wrote it, and -- once the manager has read the run as left -- the dispatch's origin and summary (`remote_runs.annotate`): the announce after a restart names the instance, node and tool call as the announce here does and quotes the task as authored, never the rendered prompt on disk; a host killed outright gets an origin made up from the spawn record and the run. - The record goes last, once the outcome has reached its conversation: a conclusion that fails, a stop under it, or an announce the scheduler refuses (`_announce_result` says so now) leaves the record, its claim released, for the next start to try again. - A run that returned here keeps its record the same way, until the manager's announce of the reply was taken: the manager dispatches with `reported_by_caller=True`, the backend then leaves the record behind with the reply instead of removing it as it returns, and the manager removes it after the announce (`remote_runs.forget_dispatch`) -- or, when the draining scheduler refused the announce, annotates it with the dispatch's origin and summary for the next start to report. A host killed between the agent's answer and the announce therefore still has the way back to the run. A caller that takes the reply as its own result (a graph node, a probe, a direct chat) dispatches without the keyword, and the record goes as it did. The record's frame count stops at the prompt: the turn's own frames, the chunks before the answer as much as the answer, are never checkpointed, so a pick-up replays the turn whole and the reply it reports is complete; a record that had moved past the answer would have resumed after it and waited forever. Not in this change: a run from the DAG lane has no spawn record to finish; it is announced from the origin the manager added to its record when it was left, or, for a host killed outright, from the run's record alone. The Tasks page reads the finished record on its next load rather than live. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Machine availability and restart checks after removing the trial switch: - New default-behavior regressions were run before the removal: 8 failed, with the enabled-value control passing. After removal, all 9 passed, including ACP/CLI recovery through the real AgentRegistry with the variable absent or set to `0`. - `uv run --no-sync pytest tests/test_acp_remote.py tests/test_node_client.py tests/test_node_server.py tests/test_node_install.py tests/test_ops_remote_path.py tests/test_permissions_remote_read.py tests/test_remote_file_tools.py tests/test_deliver_files_tool.py tests/test_raven_config_tool.py tests/test_rpc_subagents.py tests/test_shell_machine_channel.py tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py tests/test_resume_remote.py tests/test_remote_resume.py tests/test_remote_runs.py tests/test_subagent_manager.py tests/test_cli_tui_commands.py tests/test_rpc_schema_match.py tests/test_config_schema.py -n 4 --idle-ceiling-strict --cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report= --tb=short` -- 1799 passed on the combined tree, including the latest published result-retention fix. An earlier run had a missing stdio response in the minimal-node dependency test and an idle-gate hit; both passed a focused recheck, and this final complete selection passed with the strict gate. - `uv run --no-sync python scripts/coverage_gate.py --coverage-json <coverage-json> diff --base-ref origin/main --threshold 90` -- 92.92% (656/706 changed executable lines) for this PR; the gate-removal commit alone covers 2/2 new executable lines (the remainder deletes gates and updates descriptions). - `UV_NO_SYNC=1 make lint-python lint-imports lint-deps lint-types`, the commit/source-language/large-file scope gates and `git diff --check` pass. Both RPC generators' `--check` runs pass; `npm test -- scripts/gates/fixture-shape.test.mjs` from `ui-web` passes all five tests. - Real Linux machine, isolated node installations, separate dispatch and restart processes: before removal, a task started with the trial switch on was reported as an absent agent and its recovery record deleted when restarted without it. After removal, both absent and old `0` settings recover the full CLI result and read its output file by address. The absent-variable case kills the relay before host shutdown. The combined final tree was reinstalled and both scenarios repeated successfully. - With the variable absent, 12 further timing/boundary probes pass: early CLI exit and blank lines, relay loss before heartbeat with normal-call recovery, tunnel loss, ACP/CLI/OpenAI cancellation during disconnection, 11 launches admitting 8 and refusing 3, occupied tunnel port, and an unwritable prompt file. Three attachment, POSIX-command and IPv6 checks also pass; Windows command parsing is simulated, not native Windows. No test task or tunnel remains; the dedicated node directories are removed. - CONTEXT.md, configuration and RPC descriptions, transport/path docstrings, both collaboration guides and the owner's design notes were checked against the code. The guides explain the former trial switch and normal startup without it. The existing SSH, permission and limit tests continue to exercise their checks with no test-only switch setup. - `env -u RAVEN_DEBUG_MACHINES uv run --no-sync pytest -n 4 --tb=short --cov=raven --cov-branch --cov-report=json:<full-coverage-json> --cov-report=` -- 31345 passed, 230 skipped, 2 xfailed, 5 failed. A separate current-main checkout reproduces four failures: CLI color rendering on this host, LibreOffice detection, and the two simulation cases. The remaining failure was the direct-chat test stub rejecting `reported_by_caller` from the latest result-retention change; the published wrapper fix now forwards only keywords declared by its fake backend. The full suite was not rerun for that test-only correction. Final direct-chat and manager recheck: 277 passed with strict idle accounting. Full-PR changed-line coverage from this run is 96.18% (679/706). Earlier verification: - `uv run pytest tests/test_remote_runs.py tests/test_remote_resume.py tests/test_resume_remote.py tests/test_acp_client_resume.py tests/test_node_client.py tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py tests/test_cli_tui_commands.py tests/test_subagent_manager.py tests/test_rpc_bootstrap.py tests/test_cli_gateway_commands.py tests/test_acp_client_launch.py tests/test_subagent_acp.py tests/test_scope_canon.py tests/test_l4_entrances.py` -- all passed (605 passed). Covered: the record written owner-only, kept current per frame and per prompt, removed when the turn ends here and kept when this process leaves; a dispatch left at shutdown neither finished nor announced by the manager; an endpoint ended while leaving leaves the agent with the daemon (real daemon); a task picked up from the frame last read, with the watchers hearing each frame; a pick-up the machine refuses or cannot carry; an answer awaited for a request another process sent, ids carried on, no `session/cancel` while leaving; the reply read off replayed frames with the other sessions' frames ignored, a turn cut short marked partial, an unreachable machine keeping the record, a run the machine no longer has, a prompt never sent, an agent that ended while this computer was away, an answer waited out, a turn that said nothing, a pick-up beside a live connection closing its own; the manager-side conclusion: record finished and announced, an error announced and the record failed, an unreachable machine keeping the record, an agent off the table, a failure inside one pick-up not stopping the next, a run with no conversation, the once-per-process scheduling; a cli run recorded while it runs and left at shutdown, picked up whole on the real daemon behind the stand-in ssh, its pick-up's errors said (a non-zero exit, a machine that no longer has it, a link down, this computer leaving), a cancel that is not a leave ending the command with no record left; the TUI's stop saying the agents stay before the cancel and its start scheduling the pick-up; the record annotated with the dispatch's origin and summary when the manager reads it as left, the announce quoting the task as authored and naming instance, node and tool call, with and without the manager's fields; the record outliving a refused announce and a failed conclusion, claim released; a cli record read without an ACP session, an older record read with defaults; the record moving with the frames before the prompt and not past it, a dispatch's record forgotten by the manager's id with its claim, the manager dispatching with `reported_by_caller` and forgetting the record after an announce that was taken and annotating it after one that was refused; on the real daemon behind the stand-in ssh, an acp reply and a cli output returned with the record kept and the host "stopped" before the announce, the next start reading the same reply again whole, and a dispatch without the keyword leaving no record. End to end on the real daemon behind the stand-in ssh: a turn dispatched, this process "stopped" the way the gateway stops, the agent still running on the daemon, then a fresh pick-up reading its answer, finishing the record and announcing it, with the agent's process the pool's connection again. - `uv run pytest` on this tree, at an earlier revision of this change -- 29270 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine; diff coverage of this change against its base: 97.45% (153 of 157 executable changed lines), and 94.67% for the four changes against main; `lint-imports` 10 kept, 0 broken; `ty` clean. - Real machine (a registered Linux machine with Claude Code signed in, node reinstalled with this tree): process A dispatched a 25 s task, saw the prompt out after 2.4 s and the record keep up with the frames (6, then 10), then stopped the way the gateway stops 5.4 s after the dispatch, leaving the record and the agent running there; process B, started fresh, found the record and the running task, picked it up in 2.4 s, announced the agent's reply to the conversation with the machine note, finished the record, removed the run's record, held the agent's process as the pool's connection, and closing the pool ended the agent on the machine. This revision, the cli lane with the code before and after, the node reinstalled for each: a cli entry `sh -c 'echo $$ > pid.txt; sleep 20; echo done; cat'` was dispatched and the host stopped the gateway's way 2.7 s later. Before: no record, the command left running on the machine, and the next start found nothing to pick up. After: the record kept with the task as authored and the manager's origin, the command running on, and the next start picked it up in 16.8 s (the sleep), announcing `done` and `hi` with the instance, node and tool call and the machine note. The same with the link killed before the stop: recorded, left, picked up. With the machine's daemon ended between the stop and the pick-up, it ended its groups on its way out, the task's record there said exit -9, and the pick-up announced that as the run's error in 0.1 s, leaving nothing on the machine. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint`, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed The English and Chinese agent-collaboration guides explain normal startup, registered-machine setup and recovery, and the removed trial switch. ## Risk Configured machine entries are available by default; entries without a `machine` remain local. A remote-looking file path is resolved against the registry even without an environment variable; use an explicit local path such as `./a:b` for a local filename containing a colon. A formerly hidden machine entry is now available for dispatch, and an old variable set to `0` no longer disables it. Stopping Raven no longer cancels a turn running on a machine; it goes on there until the agent finishes or is cancelled, and its answer arrives in the conversation after the next start, as a sub-agent result that triggers a turn. One record per run in the raven home, owner-only, removed when the run is concluded; a machine that cannot be reached keeps it, so a run is reported at most once and never lost quietly. The adopted connection answers requests with ids past the ones the stopped process used, so a replayed frame resolves nothing sent after the restart. Rollback is a plain revert; a record left behind is then an inert file. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues #920 and #922 (merged); #921 was carried by #922. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com> Co-authored-by: Codex <noreply@openai.com>
2026-10-14 21:34:19 +08:00
def run(target, command):
asked.append(target.id)
return 0, "PYTHON=/usr/bin/python3.12\nUV=/usr/bin/uv\nFREE_KB=1048576\n"
plan = install.plan("box", run=run)
assert plan.target.id == "box" and asked == ["box"]
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
def test_a_machine_that_is_not_registered_is_refused_before_any_ssh(registry):
with pytest.raises(InstallError, match="machine 'elsewhere' is not registered"):
install.plan("elsewhere", run=lambda target, command: pytest.fail("no ssh for an unknown machine"))
# --- what an install would do -----------------------------------------------------
def test_an_install_with_the_machine_s_own_python_is_described_before_it_runs():
p = _plan(free_kb=2 * 1024 * 1024)
said = p.describe()
assert p.how == "python" and p.needed_kb == 60 * 1024
assert said.startswith(
f"Install the Raven node {bundle.install_name()} on machine 'Lab box' (box) into ~/.raven-node/{bundle.install_name()}: "
"a virtualenv of the machine's own /usr/bin/python3.12, "
)
for pin in bundle.pinned((*protocol.REQUIREMENTS, protocol.SEARCH_REQUIREMENT)):
assert pin in said
assert "About 60 MB; 2 GB free there. Nothing else on the machine changes." in said
assert HOST not in said
def test_an_install_through_uv_needs_room_for_a_python_too():
p = _plan(python="", uv="/usr/bin/uv", free_kb=None)
assert p.how == "uv" and p.needed_kb == 140 * 1024
assert "a Python 3.12 made by uv, kept beside it" in p.describe()
assert p.describe().endswith("About 140 MB. Nothing else on the machine changes.")
def test_a_machine_without_uv_gets_one_first_and_the_room_for_it_is_counted():
p = _plan(uv="", free_kb=None)
assert p.uv_from == "fetched" and p.how == "python" and p.needed_kb == 85 * 1024
said = p.describe()
assert (
f"uv {install.UV_VERSION} itself first, under ~/.raven-node/.uv (the machine has none), fetched with its own "
"/usr/bin/curl;" in said
)
assert "a virtualenv of the machine's own /usr/bin/python3.12" in said and "all through uv" in said
assert _plan(uv="", fetch="").how == "" and _plan(uv="", fetch="").uv_from == ""
def test_the_machine_s_own_python_wins_over_uv():
assert _plan(python="/usr/bin/python3.12", uv="/usr/bin/uv").how == "python"
@pytest.mark.parametrize(
("fields", "said"),
[
(
{"python": "", "uv": "", "fetch": ""},
"machine 'Lab box' (box) has no uv, and no curl or wget to fetch one with; "
f"install uv there ({install.UV_INSTALLER}), then run this again",
),
(
{"uv": "", "fetch": ""},
"machine 'Lab box' (box) has no uv, and no curl or wget to fetch one with; "
f"install uv there ({install.UV_INSTALLER}), then run this again",
),
(
{"free_kb": 10 * 1024},
"~/.raven-node on machine 'Lab box' (box) has 10 MB free and a node needs about 60 MB; "
"free some space or choose another directory with --dir",
),
(
{"python": "", "free_kb": 100 * 1024},
"~/.raven-node on machine 'Lab box' (box) has 100 MB free and a node needs about 140 MB; "
"free some space or choose another directory with --dir",
),
(
{"uv": "", "free_kb": 70 * 1024},
"~/.raven-node on machine 'Lab box' (box) has 70 MB free and a node needs about 85 MB; "
"free some space or choose another directory with --dir",
),
({"free_kb": 60 * 1024}, None),
({"free_kb": None}, None),
],
)
def test_an_install_that_cannot_go_ahead_says_why(fields, said):
assert install.refusal(_plan(**fields)) == said
_UV_ENV = 'UV_CACHE_DIR="$dir/.uv-cache" UV_PYTHON_INSTALL_DIR="$dir/.uv-python"'
# node_dir made owner-only only when the step makes it; one that is there is left as it is.
_MADE = '[ -d "$dir" ] || { mkdir -p "$dir" && chmod 700 "$dir"; }; '
def test_the_steps_build_a_virtualenv_with_uv_then_fill_it_with_uv():
steps = install.steps(_plan(python="/opt/py 3.12/bin/python3", uv="/opt/uv"))
assert [name for name, _ in steps] == ["virtualenv", "packages", "search", "code", "path", "check"]
head = "dir=~/.raven-node; id=" + bundle.install_name() + '; node="$dir/$id"; uv=/opt/uv; '
assert all(command.startswith(head) for _, command in steps)
commands = dict(steps)
assert commands["virtualenv"] == (
head + _MADE + f'{_UV_ENV} "$uv" venv --quiet --python \'/opt/py 3.12/bin/python3\' "$node"'
)
pip = f'{_UV_ENV} "$uv" pip install --quiet --python "$node/bin/python"'
assert commands["packages"] == head + f"{pip} " + " ".join(bundle.pinned(protocol.REQUIREMENTS))
assert "--default-index" not in " ".join(commands.values()), "no index configured for pip: uv's own default"
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
assert "UV_DEFAULT_INDEX" not in " ".join(commands.values()), "and nothing is read off a step's stdin for one"
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
assert "python -m pip" not in " ".join(commands.values()) and " -m venv " not in " ".join(commands.values())
(rg,) = bundle.pinned((protocol.SEARCH_REQUIREMENT,))
assert commands["search"].count(rg) == 2 and commands["search"].count(pip) == 2
assert f"--extra-index-url {protocol.PUBLIC_INDEX}" in commands["search"]
assert commands["search"].endswith(">/dev/null 2>&1 && echo RG=1 || echo RG=0")
assert commands["code"] == head + 'mkdir -p "$node/lib" && tar -xzf - -C "$node/lib"'
# ``-I``: the machine user's PYTHONPATH or PYTHONHOME reaches neither the
# check nor, with the same flag on the launch line, the node.
assert commands["check"] == head + '"$node/bin/python" -I -m raven.node --version'
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
def test_the_node_is_told_the_version_its_directory_is_named_by():
"""A node runs this code from ``lib/``, not as an installed distribution, so it has no version of its own to
read; the install writes the one the host named its directory by beside the code."""
commands = dict(install.steps(_plan(install_name="0.2.4-abcdef123456")))
assert commands["path"].endswith(' "$node/lib" 0.2.4')
assert '"raven-" + version + ".dist-info"' in commands["path"] and "Version: " in commands["path"]
assert dict(install.steps(_plan(install_name="0.0.0+local-abcdef123456")))["path"].endswith(
' "$node/lib" 0.0.0+local'
)
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
def test_uv_makes_its_python_inside_the_node_dir():
commands = dict(install.steps(_plan(python="", uv="/root/.local/bin/uv")))
assert commands["virtualenv"].endswith(
f'uv=/root/.local/bin/uv; {_MADE}{_UV_ENV} "$uv" venv --quiet --python 3.12 "$node"'
)
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
_TAKE_INDEX = "IFS= read -r UV_DEFAULT_INDEX && export UV_DEFAULT_INDEX && "
def test_the_index_pip_is_configured_with_is_read_off_the_step_s_stdin_and_named_in_the_plan():
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
p = _plan(index="https://mirror.example/simple")
commands = dict(install.steps(p))
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
head = "dir=~/.raven-node; id=" + bundle.install_name() + '; node="$dir/$id"; uv=/usr/bin/uv; '
pip = f'{_UV_ENV} "$uv" pip install --quiet --python "$node/bin/python"'
assert commands["packages"] == head + _TAKE_INDEX + pip + " " + " ".join(bundle.pinned(protocol.REQUIREMENTS))
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
# The search package tries the machine's index first, then that index with the public one beside it.
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
assert commands["search"].startswith(head + _TAKE_INDEX + "{ " + pip + " ")
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
assert commands["search"].count(pip) == 2 and f"--extra-index-url {protocol.PUBLIC_INDEX}" in commands["search"]
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
assert not any("mirror.example" in command for command in commands.values()), "no command line names it"
assert not any("--default-index" in command for command in commands.values())
others = [name for name in commands if name not in ("packages", "search")]
assert not any("UV_DEFAULT_INDEX" in commands[name] for name in others), "only the steps that run uv pip read it"
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
assert "from https://mirror.example/simple (the index pip is configured with there), and this Raven's code" in (
p.describe()
)
assert "from PyPI (pip is configured with no other index there), and this Raven's code" in _plan().describe()
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
def test_a_sign_in_inside_the_index_url_reaches_uv_only_on_stdin_never_a_command_line_or_a_sentence(small_tarball):
"""pip takes `https://user:password@host/simple`. A command line on the machine is readable by every account
there (``/proc/<pid>/cmdline``), and this computer's ssh argv by every account here; the plan is printed before
the confirmation, and under --yes."""
index = "https://user:s3cret-pw@mirror.example:8443/simple"
p = _plan(index=index)
machine = Machine()
done = install.install(p, run=machine.run, push=machine.push)
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
assert done.rg is True
assert not any("s3cret-pw" in command or "mirror.example" in command for command in machine.sent)
names = {command: name for name, command in install.steps(p)}
assert [(names[command], data) for command, data in machine.pushed_by] == [
("packages", index.encode() + b"\n"),
("search", index.encode() + b"\n"),
("code", small_tarball),
], "the two steps that run uv pip read the whole URL, one line, off stdin; the code step its tarball"
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
said = p.describe()
assert (
"from https://mirror.example:8443/simple (the index pip is configured with there, shown without its sign-in)"
in said
)
assert "s3cret-pw" not in said and "user:" not in said
failed = install._failed(
p, "packages", 2, "error: Failed to fetch: `https://user:s3cret-pw@mirror.example:8443/simple/loguru/`"
)
assert "s3cret-pw" not in failed and "https://mirror.example:8443/simple/loguru/" in failed
assert install._shown_index("https://mirror.example/simple") == "https://mirror.example/simple"
assert install._shown_index("http://u:p@[::1]:8080/simple") == "http://[::1]:8080/simple"
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
# ---- E2: a port pip's configuration holds that is not a port ---------------------------------
@pytest.mark.parametrize(
("url", "shown"),
[
("https://u:pw@mirror.example:84a3/simple", "https://mirror.example:84a3/simple"),
("https://u:pw@mirror.example:99999/simple", "https://mirror.example:99999/simple"),
("https://mirror.example:84a3/simple", "https://mirror.example:84a3/simple"),
("https://u:p@ss@mirror.example/simple", "https://mirror.example/simple"),
("https://u@[::1]:x/simple", "https://[::1]:x/simple"),
],
ids=["not a number", "past 65535", "no sign-in", "an @ in the password", "ipv6"],
)
def test_an_index_is_shown_without_its_sign_in_whatever_its_port_says(url, shown):
assert install._shown_index(url) == shown
@pytest.mark.parametrize("port", ["84a3", "99999"], ids=["not a number", "past 65535"])
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
def test_an_index_whose_port_is_not_a_port_is_refused_in_one_sentence_by_an_install_before_anything_runs(
registry, port
):
"""Only an install needs the index, so only an install refuses it: the look takes the machine as it is."""
looked: list[str] = []
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
def run(target: remote.Machine, command: str) -> tuple[int, str]:
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
looked.append(command)
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
return 0, (
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
"PYTHON=/usr/bin/python3.12\nUV=/usr/bin/uv\nUV_VERSION=0.13.0\n"
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
f"INDEX=https://user:s3cret-pw@mirror.example:{port}/simple\nFREE_KB=2097152\n"
)
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
p = install.plan("box", run=run)
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
said = (
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
"the package index pip is configured with on machine 'Lab box' (box) has a port that is not a number "
f"from 0 to 65535 (https://mirror.example:{port}/simple); correct index-url in pip's configuration there, "
"or PIP_INDEX_URL"
)
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
assert install.refusal(p) == said
machine = Machine()
with pytest.raises(InstallError) as raised:
install.install(p, run=machine.run, push=machine.push)
assert str(raised.value) == said
assert machine.sent == [] and len(looked) == 1, "nothing ran on the machine but the look"
def test_a_machine_that_has_this_node_is_found_so_and_pruned_whatever_port_its_index_names(registry):
"""Neither the look nor a prune needs the index, so a bad port in pip's configuration stops neither:
install-node says the node is already there, and --prune runs."""
p = install.plan(
"box",
run=lambda target, command: (0, "PRESENT=1\nUV=/usr/bin/uv\nINDEX=https://u:pw@mirror.example:84a3/simple\n"),
)
assert p.present
assert install.prune(p, run=lambda target, command: (0, "PRUNED=0.1-old\n")) == ("0.1-old",)
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
def test_an_index_whose_port_is_a_port_or_none_goes_ahead(registry):
"""An empty port is the scheme's default (RFC 3986), not a port that is not one."""
for index in (
"https://u:pw@mirror.example:8443/simple",
"https://mirror.example/simple",
"https://u:pw@mirror.example:/simple",
):
p = install.plan("box", run=lambda target, command, index=index: (0, f"UV=/usr/bin/uv\nINDEX={index}\n"))
assert p.index == index and "pw" not in p.describe()
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
_MIRROR = "https://mirror.example/simple"
@pytest.mark.parametrize(
("version", "fetch", "uv_from"),
[
("0.4.23", "/usr/bin/curl", "machine"),
("0.13.0", "", "machine"),
("0.4.22", "/usr/bin/curl", "fetched"),
("", "/usr/bin/wget", "fetched"),
("0.4.22", "", ""),
("", "", ""),
],
ids=[
"the first that reads it",
"a later one",
"older, fetchable",
"unreadable, fetchable",
"older, nothing to fetch with",
"unreadable, nothing to fetch with",
],
)
def test_a_machine_uv_that_would_ignore_the_configured_index_is_not_the_one_used(version, fetch, uv_from):
"""UV_DEFAULT_INDEX arrived in uv 0.4.23 (with --default-index). An older uv ignores the variable and installs
from PyPI while the plan names the mirror, so with an index configured the pinned uv is fetched instead, and
with nothing to fetch it with the install says why."""
p = _plan(index=_MIRROR, uv="/usr/bin/uv", uv_version=version, fetch=fetch)
assert p.uv_from == uv_from
commands = dict(install.steps(p)) if uv_from else {}
if uv_from == "machine":
assert "uv" not in commands and "uv=/usr/bin/uv; " in commands["packages"]
if uv_from == "fetched":
assert "uv" in commands and 'uv="$dir/.uv/uv"; ' in commands["packages"]
said = f"uv {version}" if version else "a uv that does not say which version it is"
assert (
f" uv {install.UV_VERSION} itself first, under ~/.raven-node/.uv (the machine has {said}, and only uv "
f"0.4.23 or later installs from the index pip is configured with), fetched with its own {fetch};"
) in p.describe()
assert p.needed_kb == install._NODE_KB + install._UV_KB
if not uv_from:
said = f"uv {version}" if version else "a uv that does not say which version it is"
assert install.refusal(p) == (
f"machine 'Lab box' (box) has {said}, and only uv 0.4.23 or later installs from the package index pip is "
f"configured with there; with no curl or wget to fetch uv {install.UV_VERSION}, upgrade uv there "
f"({install.UV_INSTALLER}), then run this again"
)
def test_a_uv_an_earlier_install_fetched_is_used_when_the_machine_s_own_would_ignore_the_index(registry):
"""The next install-node after one that passed the machine's uv over finds that uv first again; the one the
earlier install fetched under node_dir is reported beside it and used, so nothing is fetched twice and a
machine that has since lost its curl is not refused."""
def look(*, index: str, own_version: str = "0.13.0") -> Plan:
said = (
"PYTHON=/usr/bin/python3.12\nUV=/usr/local/bin/uv\nUV_VERSION=0.4.22\n"
"NODE_UV=/home/worker/.raven-node/.uv/uv\n"
+ (f"NODE_UV_VERSION={own_version}\n" if own_version else "")
+ "FETCH=/usr/bin/curl\n"
+ (f"INDEX={index}\n" if index else "")
+ "FREE_KB=2097152\n"
)
return install.plan("box", run=lambda target, command: (0, said))
p = look(index=_MIRROR)
assert (p.uv, p.uv_version, p.uv_from) == ("/home/worker/.raven-node/.uv/uv", "0.13.0", "machine")
commands = dict(install.steps(p))
assert "uv" not in commands and "uv=/home/worker/.raven-node/.uv/uv; " in commands["packages"]
assert p.needed_kb == install._NODE_KB and "itself first" not in p.describe()
assert install.refusal(dataclasses.replace(p, fetch="")) is None, "nothing to fetch: nothing is needed"
# Without an index the machine's own uv serves, as before; one under node_dir that is itself too old, or
# silent, does not help, and the pinned uv is fetched over it.
assert look(index="").uv == "/usr/local/bin/uv"
assert (look(index=_MIRROR, own_version="0.4.22").uv, look(index=_MIRROR, own_version="0.4.22").uv_from) == (
"/usr/local/bin/uv",
"fetched",
)
assert look(index=_MIRROR, own_version="").uv_from == "fetched"
@pytest.mark.parametrize("version", ["0.1.0", ""], ids=["an old uv", "a uv that says no version"])
def test_an_old_uv_still_serves_a_machine_with_no_index_configured(version):
p = _plan(uv="/usr/bin/uv", uv_version=version, fetch="")
assert p.uv_from == "machine" and install.refusal(p) is None and "itself first" not in p.describe()
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
def test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures():
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
"""An install refuses such an index; a plan that holds one all the same must not trace back either."""
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
p = _plan(index="https://u:pw@mirror.example:84a3/simple")
assert "https://mirror.example:84a3/simple (the index pip is configured with there, shown without its sign-in)" in (
p.describe()
)
assert "pw@" not in install._failed(p, "packages", 2, "error: https://u:pw@mirror.example:84a3/simple/x")
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
@pytest.mark.parametrize(
("fetch", "fetched_with"),
[
("/usr/bin/curl", f"curl -LsSf {install.UV_PINNED_INSTALLER}"),
("/usr/bin/wget", f"wget -qO- {install.UV_PINNED_INSTALLER}"),
],
)
def test_a_machine_without_uv_fetches_one_version_of_it_under_the_node_dir_and_nowhere_else(fetch, fetched_with):
steps = install.steps(_plan(uv="", fetch=fetch))
assert [name for name, _ in steps] == ["uv", "virtualenv", "packages", "search", "code", "path", "check"]
head = "dir=~/.raven-node; id=" + bundle.install_name() + '; node="$dir/$id"; uv="$dir/.uv/uv"; '
assert all(command.startswith(head) for _, command in steps)
# One version, as the packages are pinned; an unmanaged install, which
# writes no receipt under the home directory and changes no PATH.
assert install.UV_VERSION in install.UV_PINNED_INSTALLER and install.UV_PINNED_INSTALLER != install.UV_INSTALLER
assert dict(steps)["uv"] == (
head + _MADE + f'mkdir -p "$dir/.uv" && {fetched_with} | UV_UNMANAGED_INSTALL="$dir/.uv" sh >/dev/null '
'&& test -x "$dir/.uv/uv"'
)
assert "UV_INSTALL_DIR" not in dict(steps)["uv"] and "MODIFY_PATH" not in dict(steps)["uv"]
assert dict(steps)["virtualenv"].endswith('"$uv" venv --quiet --python /usr/bin/python3.12 "$node"')
# --- carrying it out --------------------------------------------------------------
def test_an_install_runs_every_step_and_measures_what_it_made(small_tarball):
machine = Machine()
done = install.install(_plan(), run=machine.run, push=machine.push)
assert done == install.Installed("~/.raven-node", bundle.install_name(), 51234, rg=True)
assert machine.sent[:-1] == [command for _, command in install.steps(_plan())]
assert machine.sent[-1].endswith('du -sk "$node" 2>/dev/null | cut -f1')
assert machine.pushed == [small_tarball]
assert not any('rm -rf "$node"' in command for command in machine.sent)
def test_an_install_without_rg_still_finishes(small_tarball):
machine = Machine(search=(0, "RG=0\n"))
assert install.install(_plan(), run=machine.run, push=machine.push).rg is False
@pytest.mark.parametrize("du", [(1, ""), (0, ""), (0, "du: cannot access\n")])
def test_a_size_that_cannot_be_read_is_left_unsaid(small_tarball, du):
machine = Machine(du=du)
assert install.install(_plan(), run=machine.run, push=machine.push).size_kb is None
@pytest.mark.parametrize(
("step", "answer", "said"),
[
(
"packages",
(1, "Collecting pillow\nERROR: No matching distribution found for pillow==99.0\n"),
"installing the Raven node on machine 'Lab box' (box) failed at the packages step: "
"ERROR: No matching distribution found for pillow==99.0",
),
(
"virtualenv",
(255, f"worker@{HOST}: Permission denied (publickey)."),
"installing the Raven node: machine 'Lab box' (box) refused the login; check the key it is registered with",
),
(
"virtualenv",
(127, "sh: 1: /usr/bin/python3.12: not found"),
"installing the Raven node on machine 'Lab box' (box) failed at the virtualenv step: "
"sh: 1: /usr/bin/python3.12: not found",
),
(
"code",
(2, "tar: Unexpected EOF in archive\n"),
"installing the Raven node on machine 'Lab box' (box) failed at the code step: "
"tar: Unexpected EOF in archive",
),
(
"path",
(1, ""),
"installing the Raven node on machine 'Lab box' (box) failed at the path step: exit 1",
),
("check", (124, ""), "installing the Raven node: machine 'Lab box' (box) did not answer in time"),
],
)
def test_a_step_that_fails_stops_the_install_and_removes_the_half_made_node(small_tarball, step, answer, said):
machine = Machine(**{step: answer})
with pytest.raises(InstallError) as caught:
install.install(_plan(), run=machine.run, push=machine.push)
assert str(caught.value) == said
assert HOST not in str(caught.value)
names = [name for name, _ in install.steps(_plan())]
ran = [name for name, command in install.steps(_plan()) if command in machine.sent]
assert ran == names[: names.index(step) + 1], "nothing runs after the step that failed"
assert machine.sent[-1].endswith('rm -rf "$node"')
@pytest.mark.parametrize(
("check", "reported"),
[
(json.dumps({"protocol": protocol.PROTOCOL, "digest": "000000000000"}), "000000000000"),
(json.dumps({"protocol": protocol.PROTOCOL + 1, "digest": "x"}), "x"),
("Traceback (most recent call last):\n{not json", "nothing"),
("", "nothing"),
],
)
def test_a_node_that_does_not_report_this_raven_s_code_is_removed(small_tarball, check, reported):
machine = Machine()
machine.check = check
with pytest.raises(InstallError) as caught:
install.install(_plan(), run=machine.run, push=machine.push)
assert str(caught.value) == (
"the Raven node installed on machine 'Lab box' (box) does not report this Raven's code "
f"({reported} instead of {bundle.digest()})"
)
assert machine.sent[-1].endswith('rm -rf "$node"')
def test_the_last_json_line_is_the_one_checked(small_tarball):
machine = Machine()
good = json.dumps({"protocol": protocol.PROTOCOL, "digest": bundle.digest()})
machine.check = f'{{"stale": true}}\nsome warning\n{good}\n'
assert install.install(_plan(), run=machine.run, push=machine.push).install_name == bundle.install_name()
def test_an_install_that_cannot_go_ahead_sends_nothing():
machine = Machine()
with pytest.raises(InstallError, match="has no uv, and no curl or wget"):
install.install(_plan(python="", uv="", fetch=""), run=machine.run, push=machine.push)
assert machine.sent == []
def test_an_install_interrupted_by_an_error_still_cleans_up(small_tarball):
machine = Machine()
def run(target: remote.Machine, command: str) -> tuple[int, str]:
machine.sent.append(command)
if " venv " in command:
raise remote.RemoteMachineError("machine 'Lab box' (box): no key")
return 0, ""
with pytest.raises(remote.RemoteMachineError):
install.install(_plan(), run=run, push=machine.push)
assert machine.sent[-1].endswith('rm -rf "$node"')
def test_prune_reports_the_nodes_it_removed():
sent: list[str] = []
def run(target: remote.Machine, command: str) -> tuple[int, str]:
sent.append(command)
return 0, "PRUNED=0.2.3-aaaaaaaaaaaa\n PRUNED=0.2.4-bbbbbbbbbbbb\nnoise\n"
assert install.prune(_plan(), run=run) == ("0.2.3-aaaaaaaaaaaa", "0.2.4-bbbbbbbbbbbb")
assert sent == [install.prune_script("~/.raven-node", bundle.install_name())]
def test_a_prune_that_failed_says_where():
with pytest.raises(InstallError, match=r"failed at the prune step: rm: cannot remove 'x': Busy$"):
install.prune(_plan(), run=lambda target, command: (1, "rm: cannot remove 'x': Busy"))
# --- the scripts, under a real shell ----------------------------------------------
def _sh(command: str, home: Path, path: str, **env: str) -> str:
done = subprocess.run(
["/bin/sh", "-c", command],
capture_output=True,
text=True,
env={"HOME": str(home), "PATH": path, **env},
cwd=home,
timeout=60,
)
assert done.returncode == 0, done.stderr
return done.stdout
def _script(path: Path, body: str) -> Path:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text("#!/bin/sh\n" + body, encoding="utf-8")
path.chmod(0o755)
return path
@pytest.fixture
def shell(tmp_path: Path):
"""A home and a PATH holding only what the scripts call, plus stand-in Pythons."""
if not Path("/bin/sh").exists():
pytest.skip("the scripts run under /bin/sh")
home = tmp_path / "home"
home.mkdir()
tools = tmp_path / "bin"
tools.mkdir()
for name in ("dirname", "df", "awk", "grep"):
found = shutil.which(name)
assert found, name
(tools / name).symlink_to(found)
return home, tools
def _lines(out: str) -> dict[str, str]:
return dict(line.split("=", 1) for line in out.splitlines() if "=" in line)
@pytest.mark.slow
def test_the_probe_finds_the_node_a_new_enough_python_uv_and_the_room(shell):
home, tools = shell
_script(tools / "python3.13", "exit 1\n") # too old
_script(tools / "python3.12", "exit 0\n")
_script(tools / "uv", "exit 0\n")
_script(tools / "curl", "exit 0\n")
digest = bundle.digest()
_script(
home / ".raven-node" / "0.1-x" / "bin" / "python",
f"echo '{json.dumps({'protocol': protocol.PROTOCOL, 'digest': digest})}'\n",
)
seen = _lines(_sh(install.probe_script("~/.raven-node", "0.1-x", digest), home, str(tools)))
assert seen["PRESENT"] == "1"
assert seen["PYTHON"] == str(tools / "python3.12")
assert seen["UV"] == str(tools / "uv")
assert seen["FETCH"] == str(tools / "curl")
assert seen["FREE_KB"].isdigit()
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
@pytest.mark.slow
def test_the_probe_says_which_version_the_uv_it_found_is_and_nothing_for_one_that_does_not_say(shell):
home, tools = shell
uv = _script(tools / "uv", 'echo "uv 0.4.22 (5b9c1fa1c 2024-10-16)"\n')
seen = _lines(_sh(install.probe_script("~/.raven-node", "0.1-x", "d"), home, str(tools)))
assert (seen["UV"], seen["UV_VERSION"]) == (str(uv), "0.4.22")
_script(tools / "uv", "exit 0\n")
assert "UV_VERSION" not in _lines(_sh(install.probe_script("~/.raven-node", "0.1-x", "d"), home, str(tools)))
@pytest.mark.slow
def test_the_probe_reports_the_uv_under_the_node_dir_beside_the_one_it_found_first(shell):
home, tools = shell
first = _script(tools / "uv", 'echo "uv 0.4.22 (5b9c1fa1c 2024-10-16)"\n')
own = _script(home / "nodes" / ".uv" / "uv", 'echo "uv 0.13.0 (x86_64-unknown-linux-gnu)"\n')
probe = install.probe_script(str(home / "nodes"), "0.1-x", "d")
seen = _lines(_sh(probe, home, str(tools)))
assert (seen["UV"], seen["UV_VERSION"], seen["NODE_UV"], seen["NODE_UV_VERSION"]) == (
str(first),
"0.4.22",
str(own),
"0.13.0",
)
first.unlink()
seen = _lines(_sh(probe, home, str(tools)))
assert (seen["UV"], seen["NODE_UV"]) == (str(own), str(own)), "found first, it is reported both ways"
own.unlink()
seen = _lines(_sh(probe, home, str(tools)))
assert "UV" not in seen and "NODE_UV" not in seen and "NODE_UV_VERSION" not in seen
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
@pytest.mark.slow
def test_the_probe_reads_the_index_pip_is_configured_with_as_pip_reads_it(shell):
home, tools = shell
_script(tools / "python3", f'exec {sys.executable} "$@"\n')
conf = home / ".config" / "pip" / "pip.conf"
conf.parent.mkdir(parents=True)
conf.write_text("[global]\nindex-url = https://mirror.example/simple\n", encoding="utf-8")
probe = install.probe_script("~/.raven-node", "0.1-x", "d")
assert _lines(_sh(probe, home, str(tools)))["INDEX"] == "https://mirror.example/simple"
# [install] over [global], as for pip's own install command; the environment over every file.
conf.write_text(
"[global]\nindex-url = https://a/simple\n[install]\nindex-url = https://b/simple\n", encoding="utf-8"
)
assert _lines(_sh(probe, home, str(tools)))["INDEX"] == "https://b/simple"
assert _lines(_sh(probe, home, str(tools), PIP_INDEX_URL="https://c/simple"))["INDEX"] == "https://c/simple"
conf.unlink()
assert "INDEX" not in _lines(_sh(probe, home, str(tools))), "nothing configured: uv's default"
@pytest.mark.slow
def test_a_node_dir_that_was_there_keeps_its_mode_and_one_the_install_makes_is_owner_only(shell, tmp_path):
home, tools = shell
for name in ("mkdir", "chmod"):
(tools / name).symlink_to(shutil.which(name) or name)
uv = _script(tools / "uv", "exit 0\n")
theirs = tmp_path / "theirs"
theirs.mkdir(mode=0o755)
(theirs / "notes.txt").write_text("the owner's own\n", encoding="utf-8")
ours = tmp_path / "ours"
for node_dir in (theirs, ours):
step = dict(install.steps(_plan(node_dir=str(node_dir), uv=str(uv), python=sys.executable)))["virtualenv"]
_sh(step, home, str(tools))
assert stat.S_IMODE(theirs.stat().st_mode) == 0o755 and (theirs / "notes.txt").is_file()
assert stat.S_IMODE(ours.stat().st_mode) == 0o700
@pytest.mark.slow
def test_the_probe_finds_the_uv_an_earlier_install_put_under_the_node_dir(shell):
home, tools = shell
_script(home / "nodes" / ".uv" / "uv", "exit 0\n")
seen = _lines(_sh(install.probe_script(str(home / "nodes"), "0.1-x", "d"), home, str(tools)))
assert seen["UV"] == str(home / "nodes" / ".uv" / "uv") and "FETCH" not in seen
@pytest.mark.slow
def test_the_probe_does_not_take_a_node_of_other_code_for_this_one(shell):
home, tools = shell
_script(
home / ".raven-node" / "0.1-x" / "bin" / "python",
f"echo '{json.dumps({'protocol': protocol.PROTOCOL, 'digest': '000000000000'})}'\n",
)
seen = _lines(_sh(install.probe_script("~/.raven-node", "0.1-x", bundle.digest()), home, str(tools)))
assert "PRESENT" not in seen and "PYTHON" not in seen and "UV" not in seen
assert seen["FREE_KB"].isdigit(), "room is measured on the nearest directory that exists"
@pytest.mark.slow
def test_the_probe_finds_a_uv_the_non_login_path_misses(shell):
home, tools = shell
_script(home / ".local" / "bin" / "uv", "exit 0\n")
seen = _lines(_sh(install.probe_script("/no/such/dir/nodes", "0.1-x", "d"), home, str(tools)))
assert seen["UV"] == str(home / ".local" / "bin" / "uv")
assert seen["FREE_KB"].isdigit()
@pytest.mark.slow
@pytest.mark.parametrize(("uv_rc", "said", "tries"), [(0, "RG=1", 1), (1, "RG=0", 2)])
def test_the_search_package_never_fails_the_install(shell, tmp_path, uv_rc, said, tries):
home, tools = shell
calls = tmp_path / "uv-calls"
nodes = tmp_path / "nodes"
uv = _script(tools / "uv", f'echo "$@" >> "{calls}"; exit {uv_rc}\n')
search = dict(install.steps(_plan(node_dir=str(nodes), install_name="0.1-x", uv=str(uv))))["search"]
assert _sh(search, home, str(tools)).strip() == said
asked = calls.read_text(encoding="utf-8").splitlines()
assert len(asked) == tries
assert asked[0].startswith("pip install --quiet --python ") and "--extra-index-url" not in asked[0]
if tries == 2:
assert f"--extra-index-url {protocol.PUBLIC_INDEX}" in asked[1]
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
def _sh_in(command: str, home: Path, path: str, data: str) -> subprocess.CompletedProcess[str]:
"""``command`` under ``/bin/sh`` with ``data`` on its stdin, as a pushed step gets it."""
return subprocess.run(
["/bin/sh", "-c", command],
input=data,
capture_output=True,
text=True,
env={"HOME": str(home), "PATH": path},
cwd=home,
timeout=60,
)
@pytest.mark.slow
def test_the_index_reaches_uv_through_its_environment_and_never_its_argv_under_a_real_shell(shell, tmp_path):
"""On the machine, uv's argv is what another account's ``ps`` shows; its environment is this user's alone."""
home, tools = shell
calls = tmp_path / "uv-calls"
# Fails the search package from the machine's index alone, so the fallback beside the public one runs too.
uv = _script(
tools / "uv",
f'echo "ARGV=$* INDEX=$UV_DEFAULT_INDEX" >> "{calls}"\n'
'case "$*" in *--extra-index-url*) exit 0 ;; *ripgrep*) exit 1 ;; esac\nexit 0\n',
)
index = "https://user:s3cret-pw@mirror.example:8443/simple"
commands = dict(
install.steps(_plan(node_dir=str(tmp_path / "nodes"), install_name="0.1-x", uv=str(uv), index=index))
)
packages = _sh_in(commands["packages"], home, str(tools), index + "\n")
search = _sh_in(commands["search"], home, str(tools), index + "\n")
assert packages.returncode == 0, packages.stderr
assert search.returncode == 0 and search.stdout.strip() == "RG=1", search.stderr
asked = calls.read_text(encoding="utf-8").splitlines()
assert len(asked) == 3, "the packages, the search package from the index, then beside the public one"
assert all(line.endswith(f" INDEX={index}") for line in asked), "uv reads the index from its environment"
assert not any("s3cret-pw" in line.split(" INDEX=")[0] for line in asked), "and no argv of uv's carries it"
assert f"--extra-index-url {protocol.PUBLIC_INDEX}" in asked[2]
# No line on stdin: the step fails, rather than install from PyPI behind the owner's back.
nothing = _sh_in(commands["packages"], home, str(tools), "")
assert nothing.returncode != 0 and len(calls.read_text(encoding="utf-8").splitlines()) == 3
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
@pytest.mark.slow
def test_prune_removes_only_other_nodes(shell, tmp_path):
home, tools = shell
nodes = tmp_path / "nodes dir"
for name in ("0.2.4-current", "0.2.3-old", "0.2.4-older"):
_script(nodes / name / "bin" / "python", "exit 0\n")
(nodes / name / "lib" / "raven").mkdir(parents=True)
_script(nodes / "notes" / "bin" / "python", "exit 0\n") # no lib/raven: not a node
(nodes / ".uv-python" / "cpython-3.12").mkdir(parents=True)
(nodes / "README").write_text("the owner's own file\n", encoding="utf-8")
out = _sh(install.prune_script(str(nodes), "0.2.4-current"), home, str(tools) + ":/bin:/usr/bin")
assert sorted(out.splitlines()) == ["PRUNED=0.2.3-old", "PRUNED=0.2.4-older"]
assert sorted(p.name for p in nodes.iterdir()) == [".uv-python", "0.2.4-current", "README", "notes"]
@pytest.mark.slow
def test_prune_of_a_node_dir_that_does_not_exist_removes_nothing(shell, tmp_path):
home, tools = shell
assert _sh(install.prune_script(str(tmp_path / "absent"), "0.1-x"), home, str(tools) + ":/bin:/usr/bin") == ""
# --- one real install, on this computer -------------------------------------------
@pytest.mark.slow
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
@pytest.mark.parametrize(
"index", [None, "https://user:pw@mirror.invalid/simple"], ids=["no index", "an index pip is configured with"]
)
def test_a_real_install_makes_a_node_that_reports_this_raven_s_code(registry, tmp_path, index):
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
if not Path("/bin/sh").exists():
pytest.skip("the install steps run under /bin/sh")
home = tmp_path / "home"
home.mkdir()
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
env = {"HOME": str(home), "PATH": "/usr/bin:/bin", **({"PIP_INDEX_URL": index} if index else {})}
fed: list[bytes] = []
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
def run(target: remote.Machine, command: str) -> tuple[int, str]:
if "pip install" in command:
# The package index is the one thing not reached from a test.
return 0, "RG=0\n" if "RG=1" in command else ""
done = subprocess.run(["/bin/sh", "-c", command], capture_output=True, text=True, env=env, cwd=home)
return done.returncode, done.stdout + (f"\n{done.stderr}" if done.returncode else "")
def push(target: remote.Machine, command: str, data: bytes) -> tuple[int, str]:
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
if "pip install" in command:
# Nor when the step is sent with the index on its stdin, as it is once pip is configured with one --
# by this test, or by a system-wide pip.conf of the computer running it.
fed.append(data)
return 0, "RG=0\n" if "RG=1" in command else ""
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
done = subprocess.run(["/bin/sh", "-c", command], input=data, capture_output=True, env=env, cwd=home)
return done.returncode, done.stdout.decode() + (f"\n{done.stderr.decode()}" if done.returncode else "")
nodes = tmp_path / "nodes"
_script(nodes / "0.0.1-old" / "bin" / "python", "exit 0\n")
(nodes / "0.0.1-old" / "lib" / "raven").mkdir(parents=True)
first = install.plan("box", node_dir=str(nodes), run=run)
assert not first.present
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
if index:
assert first.index == index, "the environment's index, over any pip.conf"
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
uv = shutil.which("uv")
if uv is None:
pytest.skip("the install goes through uv, which this computer does not have")
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
# Said to be recent enough rather than read off this computer's uv: with one older than 0.4.23 here, the
# case with an index would have the install fetch uv through a stand-in that stubs only pip install, and
# reach astral.sh. Which uv the plan uses is the other tests' business.
p = dataclasses.replace(first, python=sys.executable, uv=uv, uv_version=install.UV_VERSION)
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
done = install.install(p, run=run, push=push)
assert done.install_name == bundle.install_name() and done.rg is False and (done.size_kb or 0) > 1000
fix(node): an old uv no longer skips the index; the port check is an install's (#965) ## Summary Three follow-ups to #963 (installing a Raven node on a registered machine), from LivXue's review of it. Only `raven/node/install.py` and its test change. - **An old uv no longer skips the configured index.** #963 hands uv the index pip is configured with on the machine through `UV_DEFAULT_INDEX`, which uv reads since 0.4.23 (uv's changelog adds it there together with `--default-index`). A machine's own uv older than that ignores the variable and installed from PyPI while the plan named the mirror -- before #963 such a uv refused the `--default-index` argument instead, loudly. The probe now reports the found uv's version (`UV_VERSION`, from `uv --version`). With an index configured, a uv older than 0.4.23, or one that does not say its version, is passed over: the pinned uv is fetched under the node dir as for a machine with none, and the plan says why ("the machine has uv 0.4.22, and only uv 0.4.23 or later installs from the index pip is configured with"). With nothing to fetch it with, the install says so in one sentence naming the version. Without an index configured an old uv is used as before. - **The port check is an install's, not the look's.** #963 refused an index whose port is not a number from 0 to 65535 in `plan()`, which `install-node` calls before it reads whether the node is already there and before `--prune`; neither needs the index, so a machine that had its node and a bad port in pip's configuration could not be told so or pruned. The check is now in `refusal()`, with the same sentence; the look takes the machine as it is. - **The real install test no longer reaches the network.** Its `run` stand-in stood in for `uv pip install`; since #963 the two steps that run it go through `push` when an index is configured, and the `push` stand-in ran them for real -- on a computer with a system-wide `pip.conf` the test installed packages, against the module docstring's promise. The `push` stand-in now stands in for them too, and the test also runs with an index configured, checking both steps are fed it on stdin. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up commit, after the review of this PR: - A uv an earlier install fetched under `node_dir/.uv` is used in place of a machine uv that would ignore the index. The probe reports that uv on its own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and `plan()` swaps it in when the first is older than 0.4.23 (or silent) and the fetched one is not; without an index the machine's own uv serves as before, and one under `node_dir` that is itself too old does not help. Two tests failed first: the plan from such a probe (uses the fetched uv, no `uv` step, 60 MB, no refusal without curl), and the probe under a real `/bin/sh` reporting both uvs (then the one, then none). - The machines guide, English and Chinese: the "What lands on the machine" row for `uv` names the old-uv case, and the troubleshooting table has a row for the sentence an old uv earns. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - The docstring of `test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures` says the install, not the look, refuses the port. The real install test's `uv_version` is pinned to `UV_VERSION` rather than read off this computer's uv, so a test host with a uv older than 0.4.23 does not reach astral.sh through the stand-in that stubs only `pip install`. - `tests/test_node_install.py` -- 82 passed; with the cli, bundle, client and import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` -- 31781 passed, 3 failed (the libreoffice and two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (30/30 changed executable lines). - Real machine (a registered Linux machine; everything under this check's own directory, removed after): uv 0.4.22 from uv's release page was put first on the PATH the check's own runner gives the probe, with an index configured. Before this commit: the first install passed it over and fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv` step), and with curl and wget taken away was refused with "upgrade uv there" while 0.13.0 sat under `node_dir`. After: the same first install, then the second look found the fetched 0.13.0, planned 60 MB with no `uv` step, and went ahead without curl or wget. - `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed. Each new or changed case failed on main first: with an index configured, the real install test's packages step really ran `uv pip install` against the test's mirror and failed resolving it (now stood in for, both steps fed the index on stdin); `plan()` of a machine whose index names a port that is not a port now succeeds, `refusal()` gives the sentence, `install()` runs nothing, and a machine that has this node is found so and pruned; for a uv at 0.4.23 and later, older, and silent about its version, with and without curl or wget, the plan uses the machine's uv or fetches the pinned one and says why, or `refusal()` gives the exact sentence; an old uv still serves a machine with no index; under a real `/bin/sh` the probe reports the version a stand-in uv prints and nothing for one that prints none. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_import_cycle_budget.py -n 6` -- 202 passed. - `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 100% (26/26 changed executable lines). - `ruff check`, `ruff format --check`, `ty check raven/node/install.py` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from uv's release page was put where the probe finds a uv, and the install was given an index pointed at a port nothing listens on, so whether uv used it shows. With main's code the old uv was used and the install succeeded -- uv went to PyPI behind the plan. With this change the probe read 0.4.22, the plan fetched uv 0.13.0 and said why, the packages step failed with "Connection refused" (the index was used), and the node dir held uv 0.13.0 after. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The install module's docstring now says an old uv with an index configured is passed over; the new `Plan` field and the moved check are documented where they are. `CONTEXT.md` (the node entry) does not describe uv versions or the index and still holds. ## Risk A machine whose own uv is older than 0.4.23 and whose pip is configured with an index now gets the pinned uv fetched under the node dir (about 25 MB, as for a machine with no uv), or one sentence asking to upgrade uv when it has no curl or wget; every other machine is unchanged. A machine with a bad port in pip's configuration can again be told it has the node and be pruned; an install there is still refused with the same sentence. The change is to the install plan and one probe line; rollback is a plain revert. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the review of #963 (the observations F1, F2 and F3 left on it). --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
if index:
assert fed == [index.encode() + b"\n"] * 2, "both steps that run uv pip were fed the index, and stood in for"
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
node = nodes / bundle.install_name()
reported = subprocess.run(
[str(node / "bin" / "python"), "-m", "raven.node", "--version"],
capture_output=True,
text=True,
env=env,
cwd=home,
check=True,
).stdout
assert json.loads(reported) == {"protocol": protocol.PROTOCOL, "digest": bundle.digest()}
assert bundle.tree_digest(node / "lib" / "raven") == bundle.digest()
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963) ## Summary Three fixes to installing a Raven node on a registered machine (`raven ops connection install-node`), all in `raven/node/install.py`. The first two are nonblocking notes LivXue left on #920; the third is an older problem with the version a node names itself by. - **The index's sign-in stays off every command line.** The probe reads the package index pip is configured with on the machine, and the install handed it to uv as `--default-index <url>` inside the step's command. pip takes a sign-in inside that URL (`https://user:password@host/simple`), and the command was in this computer's ssh argv, the machine's `sh -c` argv and uv's argv; `/proc/<pid>/cmdline` there is readable by every account on the machine. The index now travels on the stdin of the two steps that run `uv pip install` (`packages`, `search`) and is read into `UV_DEFAULT_INDEX`, the variable uv reads as `--default-index`; a process's environment is readable only by its own user. A step that finds no line on stdin fails rather than install from PyPI behind the owner's back. With no index configured nothing changes: no stdin, uv's default. - **A port that is not a port is one sentence, not a traceback.** `_shown_index`, which drops the sign-in from the URL for every sentence, read `urlsplit(url).port`, which raises `ValueError` for a port that is not a number or is past 65535; `install-node` catches only `InstallError`, so the person saw a traceback. The display now cuts the sign-in from the text after the authority's last `@` (as `urlsplit` finds it) and never reads `.port`, and the look at the machine refuses such an index before anything is installed: "the package index pip is configured with on machine ... has a port that is not a number from 0 to 65535 (<url without sign-in>); correct index-url in pip's configuration there, or PIP_INDEX_URL". An empty port is the scheme's default (RFC 3986) and goes ahead. - **A node names its own version.** A node runs this code from `lib/`, not as an installed distribution, so `importlib.metadata` found no `raven` there: `raven.__version__` fell back to `0.0.0+unknown`, and with it the socket, lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and the version the node reports at the handshake, which the machine listing shows -- while the node's directory is named by the host's version. The `path` step now also writes a distribution record, `lib/raven-<version>.dist-info/METADATA`, with the version the directory is named by, so the node reads the same version and names itself as the host named it. It sits outside `lib/raven`, so the digest is unchanged. This needs no change to `raven/node/server.py` or `raven/node/bundle.py`. A node installed before this keeps its old names until it is reinstalled; any change to the code changes its digest and asks for a reinstall anyway. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed. Each new or changed case failed on main first (12 failed there): no step command names the index, its host or its sign-in, and `install()` feeds the whole URL as one line on stdin to exactly the `packages` and `search` steps (the `code` step still gets the tarball); under a real `/bin/sh` with a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the packages step and both tries of the search package (the machine's index, then beside the public one) see the index in their environment and never in argv, and with nothing on stdin the packages step fails before uv runs; `_shown_index` drops the sign-in for a port that is not a number, one past 65535, an `@` inside the password and an IPv6 host, without raising; `plan()` refuses an index whose port is not a number or is past 65535 with the exact sentence above and runs nothing after the look, while a numeric port, no port and an empty port go ahead; a plan holding such an index still describes itself and its failures; the `path` step passes the version the install name begins with; and the one real install on this computer (a uv virtualenv of this interpreter, this code unpacked into it) now has the node report `raven.__version__` and `bundle.install_name()` equal to this computer's install name under `-I`. - `.venv/bin/python -m pytest tests/test_node_install.py tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py tests/test_subagent_acp_remote.py tests/test_import_cycle_budget.py -n 6` -- 334 passed. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` -- 100% (16/16 changed executable lines). `ty check raven/node/install.py`, `ruff check` and `ruff format --check` clean; `check_commit_messages.py`, `check_source_language.py` and `check_large_files.py` on `origin/main..HEAD` clean. - `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172 skipped, 2 xfailed, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. - Real machine (a registered Linux machine whose pip is configured with an HTTPS mirror, installed into a node directory of this check's own and removed after): the machine's mirror URL was given a made-up sign-in, a node was installed, and the process lines on the machine and on this computer were sampled every 0.3 s while it ran. With main's code: the sign-in was in a process line in 22 of 46 samples, in `sh` and `uv` on the machine and in `ssh` here; the node reported version `0.0.0+unknown` at the handshake and its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With this change: 0 of 68 samples; the node installed, answered the handshake with this code's digest and version `0.2.4`, its `run/` held `0.2.4-<digest>.*`, and its own `raven.__version__` and install name matched its directory. With the index pointed at a port nothing listens on, the install failed at the `packages` step with "Connection refused" -- uv took the index from its environment and did not fall back to PyPI -- and the sentence held no sign-in. Nothing of this check was left on the machine. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing text changes beyond the new refusal sentence; the install's module docstring and the `Plan.index` field say where the index travels and what the version record is for. `CONTEXT.md` (the node is installed "in a directory named by the code's version and digest") was checked and still holds. ## Risk The index now reaches uv only through its environment, which only the machine user can read; nothing else about the install changes. uv has read `UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv that understood the flag understands the variable. A machine whose pip configuration names a port that is not a number now gets one sentence instead of an install that would fail at the packages step. The version record is a few lines beside the code, outside the digested package, so a node of this code reports the same digest. Rollback is a plain revert; a node installed with the record keeps working with older code, which does not read it. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows the nonblocking notes on #920 (the install's package index on the command line, and the traceback for a port that is not a number). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
# The node reads the version its directory is named by, and so names itself as the host named it: its
# socket and lock under run/, and the version it reports at the handshake.
named = subprocess.run(
[
str(node / "bin" / "python"),
"-I",
"-c",
"import raven; from raven.node import bundle; print(raven.__version__); print(bundle.install_name())",
],
capture_output=True,
text=True,
env=env,
cwd=home,
check=True,
).stdout.split()
assert named == [bundle.install_name().rsplit("-", 1)[0], bundle.install_name()]
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920) ## Summary First of three changes that let a Raven dispatch work to the machines in its machine registry on a design where the machine owns the work: a Raven daemon runs there, the ssh is only the way to it, and an ssh that goes mid-task does not end the task. This one puts the daemon on a machine and keeps the link to it; nothing dispatches to a machine yet. The next change runs acp sub-agents through it, the one after reads and writes files there by address. It follows #893 (the ssh line and the remote working directory) and replaces #897 and #901, closed when the design changed: their code is in here, reworked to the daemon. After the review's second round, four more things hold: a page read (`read_file`, here and on a machine) reads the file a chunk at a time and decodes only the page's own lines, so no one step holds the interpreter for the whole file and the file is never whole in memory; the whole-file calls on a machine go through one gate (readers together, a writer alone, a waiting writer first), so a read never answers with the old page and the new digest; the install reads the index pip is configured with on the machine the way pip does, names it in the plan and hands it to `uv pip install`; and a machine without uv gets one pinned version as an unmanaged install, which writes no receipt under the home directory and changes no PATH, while a node directory that was there keeps its mode. What a daemon is (`raven/node/`): - This Raven's own code, running as a daemon on a registered machine. The host's ssh runs a relay there (`python -I -m raven.node --attach --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session to the daemon's unix socket under the node directory, starts the daemon when none answers (`--serve`, detached, its words in `log/serve.log`), and leaves with the ssh. The daemon leaves on its own after ten minutes with no connection and no running agent; the next relay starts it again, so nothing is registered as a service and a reboot costs one connect. It opens no network port; the socket is the registered user's own (mode 600) in `run/` (0700), or, when that path is too long for a socket, in a directory of the user's own under the temp directory (0700, checked before use: a directory of the user's, never a link, never another user's, narrowed when wider). The relay forwards only to a socket of the user's and starts no daemon beside anything else; a daemon whose socket's place holds a thing it cannot remove says so in one line and leaves rather than listen beside it. ssh login is the authentication. - It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the host drives it with `AcpClient` (over an `Endpoint` seam this change adds: a process, or a channel of a link). Protocol 3. - File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`, `edit`, `pull`) run Raven's own file tools there, fenced to the `roots` each call names, so a remote read comes back exactly as a local one would. A write may carry `expect`, the digest the file had when it was read, and is refused with `CHANGED` when it no longer matches. A page read with its digest, a stat with one, a chunk and a checked write run in a thread of the daemon's, writers one at a time under a lock, so the loop that answers the heartbeat and forwards every agent's frames is not held for the length of a large file. - Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`): the command named is started in the machine user's login environment (captured once by the daemon itself, under `bash` or `zsh` `-lic`), in its own process group, under a task id of the host's. Its ACP frames travel on a numbered channel of the connection that holds it, and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`: `task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the files 0600, as the host's own journal is) as they arrive. A connection that goes leaves the agent running; `agent/attach` on a later connection replays the record from the frame the host last read and takes the task over, and a finished task is replayed from its record alone, even by a daemon that never ran it. `agent/cancel` ends the whole process tree: the tree is read off `ps` before the first signal, so a command the agent put in a session of its own (Claude Code's shell tool does) goes too. `task/list` names the tasks running or remembered; records are kept seven days. `node/shutdown` ends agents only under `--stdio`, the one mode that ties them to the connection (tests, and a fallback). - Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes a session directory and says where it is as the machine resolves it. - Small on purpose: it imports the file tools and nothing that talks to a model, reads a config or opens a port; beside Raven's code the machine needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has one (else uv brings one), loguru and pillow (about 55 MB, more with a fetched uv or Python), ripgrep-bin when its index has it. A test runs the node with nothing importable beyond the standard library, its own code and those two packages, and asks every file call, so the next import added under `raven/agent/tools` is caught here rather than on every installed node. The same code at both ends (`raven/node/bundle.py`): the code is named by a digest of the package's files, and `node/hello` compares it, so a daemon of other code is refused with the command that updates it. An install lives in `<node_dir>/<version>-<digest>`. The host side (`raven/node/client.py`): - One link per machine, shared by every caller in the process; a link idle for ten minutes with no agent on it and no call on its way to it is closed the next time any link is asked for. A heartbeat every 5 s; one unanswered for 15 s means the link is lost. - A lost link no longer fails the agents on it. Each endpoint waits, holding what the host writes meanwhile (up to 1000 frames); the link is tried again on a backoff (2, 4, 8, 16 s) for as long as any agent waits; a link that comes back attaches every task again from the frame the host last read, sends what waited and any cancel asked in between. Only a task the machine no longer knows ends an endpoint, in one sentence. A link closed on purpose just closes: the daemon keeps its agents, and `tasks()` on a later link lists them. The listing says "link lost: ...; N tasks continue on the machine; reconnecting". - Every failure is one sentence that names the machine and what to do, never its address: not installed (with the install command), unreachable, login refused, other code, a refused handshake, a call that ran out of time, or the daemon's own last word. The registry's package (`raven/ops/remote_path.py`) tells a remote path from a local one (a UNC or extended Windows path, and on a Windows host a drive-relative one, are this computer's), resolves it against the registry, and lends the tools the pool through one seam (`use_nodes`, `nodes`) -- the tools must not import the node package, which imports them. `raven/acp_client/remote.py` keeps the ssh command, the machine lookup, the posix wrapping of a remote command and the one-sentence reading of an ssh failure from #893, and loses the launch command, the env preamble and the session-directory ssh that #897 would have used: the daemon does those. The ssh command is handed to the connection as an argv, and `AcpClient.launch` takes one as it is, so no host's own grammar reads it again (a Windows host's parser split the line into twelve words). What reaches a model is a path on the machine written the way scp writes it, address included, because that is how a file there is named to the tools; the way in -- the key path, ssh's options, ssh's own messages -- reaches no config file, log line or model, and an error names a machine by id and name only. The lane stays debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no daemon is started and no install is planned, each saying why. Installing (`raven ops connection install-node <id>`, `raven/node/install.py`): one look at the machine (the install, its uv or a curl or wget to fetch one with, a Python 3.12 if it has one, the free space), then a question, then everything through uv, as the repository's dependency rule asks: a uv fetched under the node directory from its installer when the machine has none, a virtualenv uv makes of the machine's Python 3.12 or of one it brings, the pinned packages with `uv pip`, the code as a tarball on ssh's stdin, each step one ssh call and a failed step removing the half-made install; the node directory is made 0700. A machine with neither uv nor a way to fetch one is refused with the installer's address. `--dir` keeps installs elsewhere and records it on the row; `--prune` removes other versions. Nothing else on the machine changes. `raven ops connection doctor` reports the registry as before. Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner in the import contract (`pyproject.toml`, `lint-imports`); the repo layout table gets its row; the gateway, the TUI and the RPC entrance close the node pool with the ACP pool on shutdown. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `uv run pytest tests/test_node_bundle.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_install.py tests/test_acp_client_teardown.py tests/test_acp_remote.py tests/test_ops_remote_path.py tests/test_cli_ops_connection_commands.py tests/test_ops_connection_add.py tests/test_ops_transport.py tests/test_subagent_acp.py tests/test_acp_client*.py tests/test_cli_gateway*.py` -- 859 passed on this tree; diff coverage of this change's lines from these groups alone passes the 90% gate. Covered, beside what #901's tests covered (parity of every file call with the tool's own answer, the fence, each malformed field, the handshake, the stdio loop, the install's scripts and one real install on this computer, through uv): - The socket's directory is the user's own or not used (a file, a link and another user's directory refused; wider bits narrowed); a daemon refused its socket's place says so and leaves; the relay forwards to nothing that is not the user's and starts no daemon beside it; a task's records are 0700 and 0600; a ping is answered while a file's digest is taken; a call on its way keeps its link; the ssh argv reaches the connection as it is and `AcpClient.launch` takes an argv; the installer fetches uv when the machine has none (curl or wget) and refuses a machine with neither; a drive-relative path is this computer's on a Windows host and scp's elsewhere, an extended or UNC path this computer's anywhere; the node answers every file call with only its requirements importable. - A task is recorded as it runs and goes on when its connection ends; a second connection attaches from the frame the host had not read and hears the exit; a finished task is replayed from its record by a daemon that never ran it; a record left "running" by a daemon that died reads as lost; an attach for a task nobody knows is refused by code; the task list and the sweep of old records; shutdown from a host leaves a daemon's tasks running. - The daemon on its socket: two connections in turn, the agent surviving the first, the daemon leaving when idle and leaving at once when another serves the same directory; `--serve` and `--attach` need `--dir`. - The link, over a stand-in `ssh` that hands the line to a real login shell and starts the real relay and daemon: a request made while the link is down is answered after the link is back, on the same daemon; a daemon told to stop takes its agents and the turn says so; a link closed on purpose leaves the agent for the next link to list, attach and cancel; a heartbeat lost parks the endpoint and the reconnect attaches it again; a task the machine no longer knows ends its endpoint in one sentence; an attach the link cannot carry parks every waiting endpoint again; held frames are capped; a file fetched whole and checked, and refused outside the roots. - The relay in this process: it starts a daemon when none answers, bridges bytes both ways, reaches the same daemon again without starting another, and gives up in one sentence when none comes up. - Cancel ends a child the agent put in a session of its own; the tree is read off `ps`; the daemon's own group is never signalled. - `uv run pytest --collect-only` -- no import error; `lint-imports` -- 10 kept, 0 broken. The daemon's two test modules skip on win32 before importing it (fcntl, unix sockets), and the two POSIX teardown tests lend a SIGKILL where the platform has none. - `uv run pytest` on the final tree of the three changes, which includes this one -- 29217 passed, 121 skipped, 3 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs` and `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`, which fail the same way on main on this machine. The groups above also pass under `--idle-ceiling-strict`, CI's idle ceiling. - Real machines (the final tree of the three changes, two registered Linux machines, a scratch registry and `RAVEN_HOME`): - Install through uv on both: on the machine without uv, uv fetched with the machine's curl under the node directory, then a virtualenv of the machine's own Python 3.12, 16 s in all; on the machine whose home disk was full, under `/tmp`, with a Python uv brought, 11 s. Both nodes answered the handshake with this code's digest. - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms; `agent/launch` 13 ms; a request's round trip 12 ms. - With a real Claude Code running a 25 s shell loop: the host's ssh killed mid-turn, the link read as lost in 5 s, the turn finished 27 s after the kill with the agent's reply, 25 of 25 lines written, the same daemon on both sides of the gap; a cancel left 0 processes 3 s later; three sessions at once answered in 8 s; with every link closed the daemon stayed with its records and no agent process remained. - The second round's four notes (`tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_node_client.py tests/test_read_file_image.py` -- 337 passed, 100% diff coverage of that commit's lines): a page is the lines `splitlines` gives whatever the chunk edge, over every break it knows; a 120 MiB file pages with every ping answered within 0.25 s; a read with a slowed digest beside a write of the same file answers with the old page and the old digest and the write lands after; a waiting writer goes before new readers; the probe reads `pip.conf` and `PIP_INDEX_URL` as pip does under a real shell, the plan names the index and the install hands it to uv; the uv step fetches one pinned version unmanaged; a node directory that was there keeps its mode and one the install makes is owner-only. On a registered Linux machine with the node reinstalled: a page read of a 1 GiB file there beside pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads sent during a write all answered with a page and a digest of one version; the probe found the mirror that machine's pip is configured with; an install into a fresh node directory, forced down the fetched path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under the home directory untouched and the directory at 700. - The full review's five findings (`tests/test_node_client.py tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the seven new tests failing on the code before the fix, diff coverage of that commit's lines past the gate): a link the heartbeat has not yet seen die, replaced by an ordinary call, brings its parked agents along (`agent/attach` on the new link, nothing left parked); a replay the new link cannot carry keeps every frame it did not take, parks the endpoint being replayed and every one not yet reached, and the link is tried again; a cancel asked on a dead link, or failing on the new one, stays wanted until a link delivers it, and a refusal clears it; two installs on one long node directory get sockets apart; a launch still capturing the login environment is counted, so a second one at the limit is refused before the first spawns. On a registered Linux machine, one script run before and after the fix (the local relay killed with the heartbeat kept out of the way, then a plain call; a cancel asked in that gap; eleven launches at once against the limit of eight): before, the agent stayed parked and never answered, the cancelled task ran on with its record saying running, nine launches started and ten tasks ran there; after, the call took 0.3 s and the agent answered through the new link, the task ended within the reattach with its record saying so, eight started and three were refused with eight running, and nothing was left running on the machine. The other half of the first finding, a relaunch that fails under the call with agents parked: on the machine, the relay killed and the relaunch made to fail once, before the agent was still parked 30 s after the machine was reachable again and no reconnect was under way; after, the reconnect the heartbeat loss schedules runs here too, the agent was attached again on the first retry after the machine answered and answered through the restored link. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` on the branch range, `commitlint` on the commit and the prospective squash header, `ruff` -- all clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed The command describes itself in `--help`; user docs follow with the change that runs sub-agents on a machine. ## Risk Opt-in and inert: nothing starts a daemon unless the debug lane is on and a caller asks for a machine, and an install runs only when the owner asks, after saying what it will do. On a machine it adds one directory under `~/.raven-node` (or the row's `node_dir`): the install, a `run/` with the socket, lock and pid of this install, a `tasks/` with the records, a `log/`; removing the directory uninstalls it. The daemon runs as the registered user, listens on a socket in a directory only that user can open (and the relay forwards to a socket of that user's only), reads and writes only inside the roots each call names, and leaves when idle. A task's record holds the agent's ACP frames, which may quote what the agent read: it is the user's alone (0700, 0600) and is removed after seven days. The way in stays in the ssh argv, in memory; no error names the address, and a model sees it only as part of a path on the machine, written the way scp writes it. Rollback is a plain revert; an installed daemon is then an inert directory that leaves once idle. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follows #893; replaces #897 and #901. --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
assert install.plan("box", node_dir=str(nodes), run=run).present, "the next look finds it"
assert install.prune(p, run=run) == ("0.0.1-old",)
# uv's cache stays beside the install, for the next one; nothing else does.
assert sorted(path.name for path in nodes.iterdir()) == [".uv-cache", bundle.install_name()]