4 Commits

Author SHA1 Message Date
Handsome-wzw
baf76335af
fix(node): an old uv no longer skips the index; the port check is an install's (#965)
## Summary

Three follow-ups to #963 (installing a Raven node on a registered
machine),
from LivXue's review of it. Only `raven/node/install.py` and its test
change.

- **An old uv no longer skips the configured index.** #963 hands uv the
index
pip is configured with on the machine through `UV_DEFAULT_INDEX`, which
uv
  reads since 0.4.23 (uv's changelog adds it there together with
`--default-index`). A machine's own uv older than that ignores the
variable
and installed from PyPI while the plan named the mirror -- before #963
such a
uv refused the `--default-index` argument instead, loudly. The probe now
reports the found uv's version (`UV_VERSION`, from `uv --version`). With
an
  index configured, a uv older than 0.4.23, or one that does not say its
version, is passed over: the pinned uv is fetched under the node dir as
for
a machine with none, and the plan says why ("the machine has uv 0.4.22,
and
only uv 0.4.23 or later installs from the index pip is configured
with").
With nothing to fetch it with, the install says so in one sentence
naming
  the version. Without an index configured an old uv is used as before.
- **The port check is an install's, not the look's.** #963 refused an
index
whose port is not a number from 0 to 65535 in `plan()`, which
`install-node`
  calls before it reads whether the node is already there and before
`--prune`; neither needs the index, so a machine that had its node and a
bad
port in pip's configuration could not be told so or pruned. The check is
now in `refusal()`, with the same sentence; the look takes the machine
as it
  is.
- **The real install test no longer reaches the network.** Its `run`
stand-in stood in for `uv pip install`; since #963 the two steps that
run it
go through `push` when an index is configured, and the `push` stand-in
ran
  them for real -- on a computer with a system-wide `pip.conf` the test
  installed packages, against the module docstring's promise. The `push`
stand-in now stands in for them too, and the test also runs with an
index
  configured, checking both steps are fed it on stdin.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Follow-up commit, after the review of this PR:

- A uv an earlier install fetched under `node_dir/.uv` is used in place
of a
machine uv that would ignore the index. The probe reports that uv on its
  own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and
`plan()` swaps it in when the first is older than 0.4.23 (or silent) and
the fetched one is not; without an index the machine's own uv serves as
before, and one under `node_dir` that is itself too old does not help.
Two
tests failed first: the plan from such a probe (uses the fetched uv, no
  `uv` step, 60 MB, no refusal without curl), and the probe under a real
  `/bin/sh` reporting both uvs (then the one, then none).
- The machines guide, English and Chinese: the "What lands on the
machine"
row for `uv` names the old-uv case, and the troubleshooting table has a
row
  for the sentence an old uv earns. `tests/test_docs_site.py` and
  `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes.
- The docstring of
`test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures`
says the install, not the look, refuses the port. The real install
test's
`uv_version` is pinned to `UV_VERSION` rather than read off this
computer's
uv, so a test host with a uv older than 0.4.23 does not reach astral.sh
  through the stand-in that stubs only `pip install`.
- `tests/test_node_install.py` -- 82 passed; with the cli, bundle,
client and
import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` --
31781
  passed, 3 failed (the libreoffice and two simulation cases, which fail
the same way on main on this computer); diff coverage against main 100%
  (30/30 changed executable lines).
- Real machine (a registered Linux machine; everything under this
check's
own directory, removed after): uv 0.4.22 from uv's release page was put
first on the PATH the check's own runner gives the probe, with an index
  configured. Before this commit: the first install passed it over and
fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look
found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv`
  step), and with curl and wget taken away was refused with "upgrade uv
there" while 0.13.0 sat under `node_dir`. After: the same first install,
then the second look found the fetched 0.13.0, planned 60 MB with no
`uv`
  step, and went ahead without curl or wget.

- `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed.
Each
new or changed case failed on main first: with an index configured, the
real install test's packages step really ran `uv pip install` against
the
test's mirror and failed resolving it (now stood in for, both steps fed
the
index on stdin); `plan()` of a machine whose index names a port that is
not
  a port now succeeds, `refusal()` gives the sentence, `install()` runs
nothing, and a machine that has this node is found so and pruned; for a
uv
at 0.4.23 and later, older, and silent about its version, with and
without
curl or wget, the plan uses the machine's uv or fetches the pinned one
and
says why, or `refusal()` gives the exact sentence; an old uv still
serves a
machine with no index; under a real `/bin/sh` the probe reports the
version
  a stand-in uv prints and nothing for one that prints none.
- `.venv/bin/python -m pytest tests/test_node_install.py
tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py
tests/test_node_client.py tests/test_import_cycle_budget.py -n 6`
  -- 202 passed.
- `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
simulation cases, which fail the same way on main on this computer);
diff
  coverage against main 100% (26/26 changed executable lines).
- `ruff check`, `ruff format --check`, `ty check raven/node/install.py`
  clean; `check_commit_messages.py`, `check_source_language.py` and
  `check_large_files.py` on `origin/main..HEAD` clean.
- Real machine (a registered Linux machine whose pip is configured with
an
HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from
uv's
release page was put where the probe finds a uv, and the install was
given
an index pointed at a port nothing listens on, so whether uv used it
shows.
With main's code the old uv was used and the install succeeded -- uv
went to
  PyPI behind the plan. With this change the probe read 0.4.22, the plan
fetched uv 0.13.0 and said why, the packages step failed with
"Connection
  refused" (the index was used), and the node dir held uv 0.13.0 after.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

The install module's docstring now says an old uv with an index
configured
is passed over; the new `Plan` field and the moved check are documented
where they are. `CONTEXT.md` (the node entry) does not describe uv
versions or
the index and still holds.

## Risk

A machine whose own uv is older than 0.4.23 and whose pip is configured
with
an index now gets the pinned uv fetched under the node dir (about 25 MB,
as
for a machine with no uv), or one sentence asking to upgrade uv when it
has
no curl or wget; every other machine is unchanged. A machine with a bad
port
in pip's configuration can again be told it has the node and be pruned;
an
install there is still refused with the same sentence. The change is to
the
install plan and one probe line; rollback is a plain revert.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

Follows the review of #963 (the observations F1, F2 and F3 left on it).

---------

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
Handsome-wzw
d102810be3
fix(node): index off command lines, a bad port in one sentence, the node's own version (#963)
## Summary

Three fixes to installing a Raven node on a registered machine
(`raven ops connection install-node`), all in `raven/node/install.py`.
The
first two are nonblocking notes LivXue left on #920; the third is an
older
problem with the version a node names itself by.

- **The index's sign-in stays off every command line.** The probe reads
the
  package index pip is configured with on the machine, and the install
handed it to uv as `--default-index <url>` inside the step's command.
pip
takes a sign-in inside that URL (`https://user:password@host/simple`),
and
the command was in this computer's ssh argv, the machine's `sh -c` argv
and
uv's argv; `/proc/<pid>/cmdline` there is readable by every account on
the
  machine. The index now travels on the stdin of the two steps that run
`uv pip install` (`packages`, `search`) and is read into
`UV_DEFAULT_INDEX`,
  the variable uv reads as `--default-index`; a process's environment is
readable only by its own user. A step that finds no line on stdin fails
  rather than install from PyPI behind the owner's back. With no index
  configured nothing changes: no stdin, uv's default.
- **A port that is not a port is one sentence, not a traceback.**
`_shown_index`, which drops the sign-in from the URL for every sentence,
read `urlsplit(url).port`, which raises `ValueError` for a port that is
not
a number or is past 65535; `install-node` catches only `InstallError`,
so
the person saw a traceback. The display now cuts the sign-in from the
text
after the authority's last `@` (as `urlsplit` finds it) and never reads
`.port`, and the look at the machine refuses such an index before
anything
is installed: "the package index pip is configured with on machine ...
has
  a port that is not a number from 0 to 65535 (<url without sign-in>);
correct index-url in pip's configuration there, or PIP_INDEX_URL". An
empty
  port is the scheme's default (RFC 3986) and goes ahead.
- **A node names its own version.** A node runs this code from `lib/`,
not as
an installed distribution, so `importlib.metadata` found no `raven`
there:
`raven.__version__` fell back to `0.0.0+unknown`, and with it the
socket,
lock and pid the daemon keeps under `run/` (`bundle.install_name()`) and
the version the node reports at the handshake, which the machine listing
shows -- while the node's directory is named by the host's version. The
  `path` step now also writes a distribution record,
`lib/raven-<version>.dist-info/METADATA`, with the version the directory
is named by, so the node reads the same version and names itself as the
host named it. It sits outside `lib/raven`, so the digest is unchanged.
This needs no change to `raven/node/server.py` or
`raven/node/bundle.py`.

A node installed before this keeps its old names until it is
reinstalled;
any change to the code changes its digest and asks for a reinstall
anyway.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `.venv/bin/python -m pytest tests/test_node_install.py` -- 69 passed.
Each
  new or changed case failed on main first (12 failed there): no step
command names the index, its host or its sign-in, and `install()` feeds
the
  whole URL as one line on stdin to exactly the `packages` and `search`
steps (the `code` step still gets the tarball); under a real `/bin/sh`
with
a stand-in uv that records its argv and its `UV_DEFAULT_INDEX`, the
packages
step and both tries of the search package (the machine's index, then
beside
the public one) see the index in their environment and never in argv,
and
  with nothing on stdin the packages step fails before uv runs;
`_shown_index` drops the sign-in for a port that is not a number, one
past
  65535, an `@` inside the password and an IPv6 host, without raising;
`plan()` refuses an index whose port is not a number or is past 65535
with
the exact sentence above and runs nothing after the look, while a
numeric
  port, no port and an empty port go ahead; a plan holding such an index
  still describes itself and its failures; the `path` step passes the
  version the install name begins with; and the one real install on this
computer (a uv virtualenv of this interpreter, this code unpacked into
it)
now has the node report `raven.__version__` and `bundle.install_name()`
  equal to this computer's install name under `-I`.
- `.venv/bin/python -m pytest tests/test_node_install.py
tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py
tests/test_node_client.py tests/test_node_server.py
tests/test_node_relay.py tests/test_subagent_acp_remote.py
tests/test_import_cycle_budget.py -n 6`
  -- 334 passed.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
--
100% (16/16 changed executable lines). `ty check raven/node/install.py`,
`ruff check` and `ruff format --check` clean;
`check_commit_messages.py`,
  `check_source_language.py` and `check_large_files.py` on
  `origin/main..HEAD` clean.
- `.venv/bin/python -m pytest -n 6` on this tree -- 31716 passed, 172
skipped,
2 xfailed, 3 failed:
`test_rpc_files.py::test_a_host_without_libreoffice_says_so`,

`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
  which fail the same way on main on this machine.
- Real machine (a registered Linux machine whose pip is configured with
an
  HTTPS mirror, installed into a node directory of this check's own and
removed after): the machine's mirror URL was given a made-up sign-in, a
node
was installed, and the process lines on the machine and on this computer
were sampled every 0.3 s while it ran. With main's code: the sign-in was
in
a process line in 22 of 46 samples, in `sh` and `uv` on the machine and
in
`ssh` here; the node reported version `0.0.0+unknown` at the handshake
and
its `run/` held `0.0.0+unknown-<digest>.lock`, `.pid` and `.sock`. With
this change: 0 of 68 samples; the node installed, answered the handshake
  with this code's digest and version `0.2.4`, its `run/` held
`0.2.4-<digest>.*`, and its own `raven.__version__` and install name
matched
its directory. With the index pointed at a port nothing listens on, the
install failed at the `packages` step with "Connection refused" -- uv
took
the index from its environment and did not fall back to PyPI -- and the
sentence held no sign-in. Nothing of this check was left on the machine.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

No user-facing text changes beyond the new refusal sentence; the
install's
module docstring and the `Plan.index` field say where the index travels
and
what the version record is for. `CONTEXT.md` (the node is installed "in
a
directory named by the code's version and digest") was checked and still
holds.

## Risk

The index now reaches uv only through its environment, which only the
machine user can read; nothing else about the install changes. uv has
read
`UV_DEFAULT_INDEX` since it gained `--default-index`, so a machine's uv
that understood the flag understands the variable. A machine whose pip
configuration names a port that is not a number now gets one sentence
instead of an install that would fail at the packages step. The version
record is a few lines beside the code, outside the digested package, so
a
node of this code reports the same digest. Rollback is a plain revert; a
node
installed with the record keeps working with older code, which does not
read
it.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

Follows the nonblocking notes on #920 (the install's package index on
the command line, and the traceback for a port that is not a number).

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-15 15:58:24 +08:00
Handsome-wzw
aa023212a0
feat(*): pick up a turn left on a registered machine when this computer stops (#934)
## Summary

Registered machines are now available without a debug environment
switch.
`RAVEN_DEBUG_MACHINES` previously defaulted to off: it kept
machine-backed
agents out of the dispatch table, refused node/installer access, hid
remote
preset destinations, and treated file paths locally while off. It was a
trial feature gate, not a verbose-logging option. At the owner's
request,
this PR removes the variable and all of those checks. Normal startup and
restart recovery use the configured machine entries; even a leftover
value
of `0` is ignored. Machine registration, SSH authentication, file
permission
prompts, and concurrency limits still apply.

Fourth change in the series that lets a Raven dispatch work to the
machines in its machine registry. #920, #921 (merged through #922) and
#922 are on main; this change stands on main alone now, and the diff is
its own.

What this does: a turn running on a registered machine survives this
computer stopping, and the next start picks it up and reports it.

- While a turn runs on a machine, this computer remembers what the
  machine does not know (`raven/acp_client/remote_runs.py`): one
  owner-only file per run under `<raven home>/machines/runs/`, with the
machine's registry id, the task id its daemon runs the agent under, the
  agent, the session key, the instance handle, the dispatch id, the ACP
  session, the id of the prompt whose answer is awaited, the ids this
  side has used, how many of the agent's frames were read here, the
  agent's answer to `initialize` and the connection's launch key. Kept
  current after every frame the connection has dealt with, removed when
  the turn ends here. No address is in it.
- When this process stops -- the gateway, `serve`, the TUI, the page
  stack -- the
  agents on machines stay with their daemons
  (`remote.leave_agents_running()` before the drain): a cancelled turn
on a machine sends no `session/cancel`, closing its connection ends the
endpoint only, and the manager leaves the dispatch's record open rather
  than finishing it as cancelled. A hard kill or `/restart` leaves the
  same record, written as the turn ran.
- At the next start -- the gateway once its generation is bound; the
`serve`, TUI and page stacks once --
`raven/agent/subagent/resume_remote.py`
picks every run up: `task/list` on the machine, `agent/attach` from the
  frame last read (`NodePool.attach`), the connection adopted into the
  pool with request ids carried on past the other process's
  (`AcpConnectionPool.adopt`, `AcpClient.continue_ids_from`), the
  prompt's answer awaited (`AcpClient.await_response`), the reply built
  as a turn ended here is -- the partial notice when the agent stopped
  early, the machine note saying it was picked up after this computer
  restarted -- then the dispatch's record finished, the registry row and
  the node claim closed, and the result announced into the conversation
  that asked, as the manager announces a turn that ended here
  (`raven/acp_client/remote_resume.py`). A machine that cannot be asked
  keeps its record for a later start; a run the machine no longer has,
  one whose agent ended while this computer was away, one whose agent
  left the agent table, and one whose turn says nothing are announced as
  errors that say so. The adopted agent process serves later dispatches
  of its entry, as one started here would.
- Seams: `AcpClient.request(sent=)` hears the id of what goes out;
  `ChannelEndpoint.watchers` hear the frames the connection has dealt
  with (`consumed`, called by the read loop); `ChannelEndpoint.stays`
  reads the leaving flag.
- A cli entry's run on a machine is recorded while it runs (kind `cli`,
  no ACP part) and left the same way, the manager reading the dispatch
  as left; the entry picks it up (`CliAgentBackend.pick_up`): the daemon
  replays the command's whole output from its first line, the exit code
  ends it, and the reply closes with the machine note.
- The record carries the task as the model wrote it, and -- once the
  manager has read the run as left -- the dispatch's origin and summary
  (`remote_runs.annotate`): the announce after a restart names the
  instance, node and tool call as the announce here does and quotes the
  task as authored, never the rendered prompt on disk; a host killed
  outright gets an origin made up from the spawn record and the run.
- The record goes last, once the outcome has reached its conversation:
  a conclusion that fails, a stop under it, or an announce the scheduler
  refuses (`_announce_result` says so now) leaves the record, its claim
  released, for the next start to try again.
- A run that returned here keeps its record the same way, until the
  manager's announce of the reply was taken: the manager dispatches with
  `reported_by_caller=True`, the backend then leaves the record behind
  with the reply instead of removing it as it returns, and the manager
  removes it after the announce (`remote_runs.forget_dispatch`) -- or,
  when the draining scheduler refused the announce, annotates it with
  the dispatch's origin and summary for the next start to report. A host
  killed between the agent's answer and the announce therefore still has
  the way back to the run. A caller that takes the reply as its own
  result (a graph node, a probe, a direct chat) dispatches without the
  keyword, and the record goes as it did. The record's frame count stops
  at the prompt: the turn's own frames, the chunks before the answer as
  much as the answer, are never checkpointed, so a pick-up replays the
  turn whole and the reply it reports is complete; a record that had
  moved past the answer would have resumed after it and waited forever.

Not in this change: a run from the DAG lane has no spawn record to
finish; it is announced from the origin the manager added to its record
when it was left, or, for a host killed outright, from the run's record
alone. The Tasks page reads the finished record on its next load rather
than live.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Machine availability and restart checks after removing the trial switch:

- New default-behavior regressions were run before the removal: 8
failed,
  with the enabled-value control passing. After removal, all 9 passed,
  including ACP/CLI recovery through the real AgentRegistry with the
  variable absent or set to `0`.
- `uv run --no-sync pytest tests/test_acp_remote.py
tests/test_node_client.py tests/test_node_server.py
tests/test_node_install.py tests/test_ops_remote_path.py
tests/test_permissions_remote_read.py tests/test_remote_file_tools.py
tests/test_deliver_files_tool.py tests/test_raven_config_tool.py
tests/test_rpc_subagents.py tests/test_shell_machine_channel.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_resume_remote.py tests/test_remote_resume.py
tests/test_remote_runs.py tests/test_subagent_manager.py
tests/test_cli_tui_commands.py tests/test_rpc_schema_match.py
tests/test_config_schema.py -n 4 --idle-ceiling-strict --cov=raven
--cov-branch --cov-report=json:<coverage-json> --cov-report= --tb=short`
  -- 1799 passed on the combined tree, including the latest published
result-retention fix. An earlier run had a missing stdio response in the
minimal-node dependency test and an idle-gate hit; both passed a focused
recheck, and this final complete selection passed with the strict gate.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 92.92% (656/706 changed executable lines) for this PR; the
gate-removal
commit alone covers 2/2 new executable lines (the remainder deletes
gates
  and updates descriptions).
- `UV_NO_SYNC=1 make lint-python lint-imports lint-deps lint-types`, the
commit/source-language/large-file scope gates and `git diff --check`
pass.
Both RPC generators' `--check` runs pass; `npm test --
scripts/gates/fixture-shape.test.mjs`
  from `ui-web` passes all five tests.
- Real Linux machine, isolated node installations, separate dispatch and
restart processes: before removal, a task started with the trial switch
on was reported as an absent agent and its recovery record deleted when
  restarted without it. After removal, both absent and old `0` settings
  recover the full CLI result and read its output file by address. The
absent-variable case kills the relay before host shutdown. The combined
  final tree was reinstalled and both scenarios repeated successfully.
- With the variable absent, 12 further timing/boundary probes pass:
early
  CLI exit and blank lines, relay loss before heartbeat with normal-call
recovery, tunnel loss, ACP/CLI/OpenAI cancellation during disconnection,
  11 launches admitting 8 and refusing 3, occupied tunnel port, and an
unwritable prompt file. Three attachment, POSIX-command and IPv6 checks
  also pass; Windows command parsing is simulated, not native Windows.
No test task or tunnel remains; the dedicated node directories are
removed.
- CONTEXT.md, configuration and RPC descriptions, transport/path
docstrings,
  both collaboration guides and the owner's design notes were checked
against the code. The guides explain the former trial switch and normal
startup without it. The existing SSH, permission and limit tests
continue
  to exercise their checks with no test-only switch setup.

- `env -u RAVEN_DEBUG_MACHINES uv run --no-sync pytest -n 4 --tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
-- 31345 passed, 230 skipped, 2 xfailed, 5 failed. A separate
current-main
  checkout reproduces four failures: CLI color rendering on this host,
LibreOffice detection, and the two simulation cases. The remaining
failure
  was the direct-chat test stub rejecting `reported_by_caller` from the
latest result-retention change; the published wrapper fix now forwards
only keywords declared by its fake backend. The full
suite was not rerun for that test-only correction. Final direct-chat and
manager recheck: 277 passed with strict idle accounting. Full-PR
changed-line coverage from this
  run is 96.18% (679/706).

Earlier verification:

- `uv run pytest tests/test_remote_runs.py tests/test_remote_resume.py
tests/test_resume_remote.py tests/test_acp_client_resume.py
tests/test_node_client.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_cli_tui_commands.py
tests/test_subagent_manager.py tests/test_rpc_bootstrap.py
tests/test_cli_gateway_commands.py tests/test_acp_client_launch.py
tests/test_subagent_acp.py tests/test_scope_canon.py
tests/test_l4_entrances.py`
-- all passed (605 passed). Covered: the record written owner-only, kept
current
per frame and per prompt, removed when the turn ends here and kept when
  this process leaves; a dispatch left at shutdown neither finished nor
  announced by the manager; an endpoint ended while leaving leaves the
  agent with the daemon (real daemon); a task picked up from the frame
  last read, with the watchers hearing each frame; a pick-up the machine
  refuses or cannot carry; an answer awaited for a request another
  process sent, ids carried on, no `session/cancel` while leaving; the
  reply read off replayed frames with the other sessions' frames
  ignored, a turn cut short marked partial, an unreachable machine
  keeping the record, a run the machine no longer has, a prompt never
  sent, an agent that ended while this computer was away, an answer
  waited out, a turn that said nothing, a pick-up beside a live
  connection closing its own; the manager-side conclusion: record
  finished and announced, an error announced and the record failed, an
  unreachable machine keeping the record, an agent off the table, a
  failure inside one pick-up not stopping the next, a run with no
  conversation, the once-per-process scheduling; a cli run recorded
  while it runs and left at shutdown, picked up whole on the real daemon
  behind the stand-in ssh, its pick-up's errors said (a non-zero exit, a
  machine that no longer has it, a link down, this computer leaving), a
  cancel that is not a leave ending the command with no record left; the
  TUI's stop saying the agents stay before the cancel and its start
  scheduling the pick-up; the record annotated with the dispatch's
  origin and summary when the manager reads it as left, the announce
  quoting the task as authored and naming instance, node and tool call,
  with and without the manager's fields; the record outliving a refused
  announce and a failed conclusion, claim released; a cli record read
  without an ACP session, an older record read with defaults; the record
  moving with the frames before the prompt and not past it, a dispatch's
  record forgotten by the manager's id with its claim, the manager
  dispatching with `reported_by_caller` and forgetting the record after
  an announce that was taken and annotating it after one that was
  refused; on the real daemon behind the stand-in ssh, an acp reply and
  a cli output returned with the record kept and the host "stopped"
  before the announce, the next start reading the same reply again
whole, and a dispatch without the keyword leaving no record. End to end
on the real
  daemon behind the stand-in ssh: a turn dispatched, this process
  "stopped" the way the gateway stops, the agent still running on the
  daemon, then a fresh pick-up reading its answer, finishing the record
  and announcing it, with the agent's process the pool's connection
  again.
- `uv run pytest` on this tree, at an earlier revision of this change --
29270 passed, 121 skipped, 3 failed:
  `test_rpc_files.py::test_a_host_without_libreoffice_says_so`,

`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
  which fail the same way on main on this machine; diff coverage of this
  change against its base: 97.45% (153 of 157 executable changed lines),
  and 94.67% for the four changes against main; `lint-imports` 10 kept,
  0 broken; `ty` clean.
- Real machine (a registered Linux machine with Claude Code signed in,
  node reinstalled with this tree): process A dispatched a 25 s task,
  saw the prompt out after 2.4 s and the record keep up with the frames
  (6, then 10), then stopped the way the gateway stops 5.4 s after the
  dispatch, leaving the record and the agent running there; process B,
  started fresh, found the record and the running task, picked it up
  in 2.4 s, announced the agent's reply to the conversation with the
  machine note, finished the record, removed the run's record, held the
  agent's process as the pool's connection, and closing the pool ended
  the agent on the machine. This revision, the cli lane with the code
  before and after, the node reinstalled for each: a cli entry
  `sh -c 'echo $$ > pid.txt; sleep 20; echo done; cat'` was dispatched
  and the host stopped the gateway's way 2.7 s later. Before: no
  record, the command left running on the machine, and the next start
  found nothing to pick up. After: the record kept with the task as
  authored and the manager's origin, the command running on, and the
  next start picked it up in 16.8 s (the sleep), announcing `done` and
  `hi` with the instance, node and tool call and the machine note. The
  same with the link killed before the stop: recorded, left, picked up.
  With the machine's daemon ended between the stop and the pick-up, it
  ended its groups on its way out, the task's record there said exit
  -9, and the pick-up announced that as the run's error in 0.1 s,
  leaving nothing on the machine.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
  and `scripts/check_large_files.py` on the branch range, `commitlint`,
  `ruff` -- all clean.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

The English and Chinese agent-collaboration guides explain normal
startup,
registered-machine setup and recovery, and the removed trial switch.

## Risk

Configured machine entries are available by default; entries without a
`machine` remain local. A remote-looking file path is resolved against
the
registry even without an environment variable; use an explicit local
path
such as `./a:b` for a local filename containing a colon. A formerly
hidden
machine entry is now available for dispatch, and an old variable set to
`0`
no longer disables it. Stopping Raven no longer cancels a turn
running on a machine; it goes on there until the agent finishes or is
cancelled, and its answer arrives in the conversation after the next
start, as a sub-agent result that triggers a turn. One record per run in
the raven home, owner-only, removed when the run is concluded; a machine
that cannot be reached keeps it, so a run is reported at most once and
never lost quietly. The adopted connection answers requests with ids
past the ones the stopped process used, so a replayed frame resolves
nothing sent after the restart. Rollback is a plain revert; a record
left
behind is then an inert file.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

#920 and #922 (merged); #921 was carried by #922.

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-14 21:34:19 +08:00
Handsome-wzw
e3d0135049
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary

First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.

It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.

After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.

What a daemon is (`raven/node/`):

- This Raven's own code, running as a daemon on a registered machine.
  The host's ssh runs a relay there (`python -I -m raven.node --attach
  --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
  when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
  so nothing is registered as a service and a reboot costs one connect.
  It opens no network port; the socket is the registered user's own
  (mode 600) in `run/` (0700), or, when that path is too long for a
  socket, in a directory of the user's own under the temp directory
  (0700, checked before use: a directory of the user's, never a link,
  never another user's, narrowed when wider). The relay forwards only to
  a socket of the user's and starts no daemon beside anything else; a
  daemon whose socket's place holds a thing it cannot remove says so in
  one line and leaves rather than listen beside it. ssh login is the
  authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
  host drives it with `AcpClient` (over an `Endpoint` seam this change
  adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
  `pull`) run Raven's own file tools there, fenced to the `roots` each
  call names, so a remote read comes back exactly as a local one would.
  A write may carry `expect`, the digest the file had when it was read,
  and is refused with `CHANGED` when it no longer matches. A page read
  with its digest, a stat with one, a chunk and a checked write run in a
  thread of the daemon's, writers one at a time under a lock, so the
  loop that answers the heartbeat and forwards every agent's frames is
  not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
  the command named is started in the machine user's login environment
  (captured once by the daemon itself, under `bash` or `zsh` `-lic`),
  in its own process group, under a task id of the host's. Its ACP
  frames travel on a numbered channel of the connection that holds it,
  and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
  files 0600, as the host's own journal is) as they arrive. A connection
  that goes leaves the agent running; `agent/attach` on a later
  connection replays the record from the frame the host last read and
  takes the task over, and a finished task is replayed from its record
  alone, even by a daemon that never ran it. `agent/cancel` ends the
  whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
  records are kept seven days. `node/shutdown` ends agents only under
  `--stdio`, the one mode that ties them to the connection (tests, and a
  fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
  a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
  model, reads a config or opens a port; beside Raven's code the machine
  needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
  one (else uv brings one), loguru and pillow (about 55 MB, more with a
  fetched uv or Python), ripgrep-bin when its index has it. A test runs
  the node with nothing importable beyond the standard library, its own
  code and those two packages, and asks every file call, so the next
  import added under `raven/agent/tools` is caught here rather than on
  every installed node.

The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.

The host side (`raven/node/client.py`):

- One link per machine, shared by every caller in the process; a link
  idle for ten minutes with no agent on it and no call on its way to it
  is closed the next time any link is asked for. A heartbeat every 5 s;
  one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
  tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
  waits; a link that comes back attaches every task again from the frame
  the host last read, sends what waited and any cancel asked in between.
  Only a task the machine no longer knows ends an endpoint, in one
  sentence. A link closed on purpose just closes: the daemon keeps its
  agents, and `tasks()` on a later link lists them. The listing says
  "link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
  never its address: not installed (with the install command),
  unreachable, login refused, other code, a refused handshake, a call
  that ran out of time, or the daemon's own last word.

The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.

Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.

Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
  -- 859 passed on this tree; diff coverage of this change's lines from
  these groups alone passes the 90% gate. Covered, beside what #901's
  tests covered (parity of every file call with the tool's own answer,
  the fence, each malformed field, the handshake, the stdio loop, the
  install's scripts and one real install on this computer, through uv):
  - The socket's directory is the user's own or not used (a file, a
    link and another user's directory refused; wider bits narrowed); a
    daemon refused its socket's place says so and leaves; the relay
    forwards to nothing that is not the user's and starts no daemon
    beside it; a task's records are 0700 and 0600; a ping is answered
    while a file's digest is taken; a call on its way keeps its link;
    the ssh argv reaches the connection as it is and `AcpClient.launch`
    takes an argv; the installer fetches uv when the machine has none
    (curl or wget) and refuses a machine with neither; a drive-relative
    path is this computer's on a Windows host and scp's elsewhere, an
    extended or UNC path this computer's anywhere; the node answers
    every file call with only its requirements importable.
  - A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
    hears the exit; a finished task is replayed from its record by a
    daemon that never ran it; a record left "running" by a daemon that
    died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
    leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
    the first, the daemon leaving when idle and leaving at once when
    another serves the same directory; `--serve` and `--attach` need
    `--dir`.
  - The link, over a stand-in `ssh` that hands the line to a real login
    shell and starts the real relay and daemon: a request made while the
    link is down is answered after the link is back, on the same daemon;
    a daemon told to stop takes its agents and the turn says so; a link
    closed on purpose leaves the agent for the next link to list, attach
    and cancel; a heartbeat lost parks the endpoint and the reconnect
    attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
    waiting endpoint again; held frames are capped; a file fetched whole
    and checked, and refused outside the roots.
  - The relay in this process: it starts a daemon when none answers,
    bridges bytes both ways, reaches the same daemon again without
    starting another, and gives up in one sentence when none comes up.
  - Cancel ends a child the agent put in a session of its own; the tree
    is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
  10 kept, 0 broken. The daemon's two test modules skip on win32 before
  importing it (fcntl, unix sockets), and the two POSIX teardown tests
  lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
  this one -- 29217 passed, 121 skipped, 3 failed:
  `test_rpc_files.py::test_a_host_without_libreoffice_says_so`,

`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
  which fail the same way on main on this machine. The groups above
  also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
  Linux machines, a scratch registry and `RAVEN_HOME`):
  - Install through uv on both: on the machine without uv, uv fetched
    with the machine's curl under the node directory, then a virtualenv
    of the machine's own Python 3.12, 16 s in all; on the machine whose
    home disk was full, under `/tmp`, with a Python uv brought, 11 s.
    Both nodes answered the handshake with this code's digest.
  - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
    `agent/launch` 13 ms; a request's round trip 12 ms.
  - With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
    after the kill with the agent's reply, 25 of 25 lines written, the
    same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
    the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
  tests/test_node_install.py tests/test_node_server.py
  tests/test_node_client.py tests/test_read_file_image.py` -- 337
  passed, 100% diff coverage of that commit's lines): a page is the
  lines `splitlines` gives whatever the chunk edge, over every break it
  knows; a 120 MiB file pages with every ping answered within 0.25 s; a
  read with a slowed digest beside a write of the same file answers with
  the old page and the old digest and the write lands after; a waiting
  writer goes before new readers; the probe reads `pip.conf` and
  `PIP_INDEX_URL` as pip does under a real shell, the plan names the
  index and the install hands it to uv; the uv step fetches one pinned
  version unmanaged; a node directory that was there keeps its mode and
  one the install makes is owner-only. On a registered Linux machine
  with the node reinstalled: a page read of a 1 GiB file there beside
  pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
  sent during a write all answered with a page and a digest of one
  version; the probe found the mirror that machine's pip is configured
  with; an install into a fresh node directory, forced down the fetched
  path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
  the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
  tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
  seven new tests failing on the code before the fix, diff coverage of
  that commit's lines past the gate): a link the heartbeat has not yet
  seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
  being replayed and every one not yet reached, and the link is tried
  again; a cancel asked on a dead link, or failing on the new one, stays
  wanted until a link delivers it, and a refusal clears it; two installs
  on one long node directory get sockets apart; a launch still capturing
  the login environment is counted, so a second one at the limit is
  refused before the first spawns. On a registered Linux machine, one
  script run before and after the fix (the local relay killed with the
  heartbeat kept out of the way, then a plain call; a cancel asked in
  that gap; eleven launches at once against the limit of eight): before,
  the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
  there; after, the call took 0.3 s and the agent answered through the
  new link, the task ended within the reattach with its record saying
  so, eight started and three were refused with eight running, and
  nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
  reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
  after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
  the commit and the prospective squash header, `ruff` -- all clean.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.

## Risk

Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

Follows #893; replaces #897 and #901.

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00