Raven-1/tests/test_node_bridge.py
Handsome-wzw c43006bf0f
feat(*): lend the host's MCP servers to an acp agent on a machine through its node (#947)
## Summary

Adds MCP lending for remote ACP agents on top of the machine and
recovery
services already merged in #920, #922 and #934. The branch includes
current
main and its default-on registered-machine behavior.

What this does: an acp entry on a registered machine is handed the MCP
servers its row grants, the way an entry on this computer is, and the
servers themselves stay here with their definitions and keys.

- The agent on the machine runs the node's own bridge as a stdio MCP
  server (`python -I -m raven.node --mcp --dir <node dir> --task <task>
  --lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte
  pumps around a connection to the daemon's socket, no MCP parsed, no
  credential held, the standard library only.
- The daemon carries the bridge's frames as a *stream* to the host
  holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6,
`raven/node/protocol.py`, `raven/node/server.py`). The daemon tells
which
  task a bridge is by the process that connected -- the peer pid off the
socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the
parent chain to the agent the daemon started, which the bridge is under
and no other task's process is; the kernel supplies that pid and a peer
cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the
task it names must be the one the connection is a process of. Two tasks
on
  one machine run as one user, so no name or file a bridge could present
would tell them apart; the connecting process does. One stream per
bridge, at most 16 per task; a frame
  travels in pieces of at most 256 KiB characters, the last marked
(`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the
  daemon relays and keeps nothing. A stream ends with either end, the
  host's link or the task, and the other end is told.
- The host opens the endpoints for a machine row exactly as for a row
here
(`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in
the host's sandbox, HTTP auth added here), binds them to the machine,
dispatch and acquired agent task
(`NodePool.lend`/`unlend`) before handing out bridge stanzas. On
`mcp/open`
the host checks the daemon-authenticated task against that binding
before
  connecting; it rechecks the grant and link after the socket opens. An
unlend, replacement grant or replaced link cannot authorize that pending
connection. The host then pumps the frames both ways
(`raven/node/client.py`).
`node/hello` names the bridge; a node serving without a daemon
(`--stdio`)
  names none, and the dispatch then goes without MCP and says so.
- The reply closes with the servers lent and the machine's node, or with
why none were. Remote lending requires Unix-socket support on the host:
on Windows it is withheld before endpoint setup, the task continues, and
the reply explains why. The roster's "injectable MCP" matches that
support
  (`session_mcp_delivered`). The cli lane on a machine still
  withholds its servers (the next change); the openai lane has no tools.

Not in this change: the cli lane's MCP file on the machine; a stream
that
survives the link going (a bridge sees its server end, the agent reports
the call failed, and the next dispatch works again); MCP for a run
picked
up after this computer restarted (#934): it reports its calls fail.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Follow-up fix 05eaba67a, after the c9de53947 review:

- The nine tests the Windows review named now pin the host fact through
one
  fixture, `unix_socket_host` (`tests/conftest.py`), which sets
  `raven.mcp.bridge.unix_sockets_available` to true and re-points every
  `raven.*` module that imported it by name; their bodies are unchanged.
- Found complete, not just the nine: the whole suite was run with
  `asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted
before collection (a pytest plugin, controller and xdist workers), at
the
  merge base `6b938a426` and at `c9de53947`, and the two failure sets
compared. 17 node ids failed at the head and not at the base: the nine,
and eight this change added that run the transport for real (the bridge
module, and two lending tests through the stand-in machine). Those eight
are guarded by host capability instead of pinned: the bridge module
skips
when `sys.platform == "win32" or not unix_sockets_available()`, the two
lending tests carry the existing `skipif(not unix_sockets_available())`.
After the fix the same run fails the same 66 node ids at the head as at
  the base: nothing this change adds or touches fails on a host without
  unix sockets.
- A bridge whose daemon side goes ends as streams end. The host can
close a
  stream while a frame of the agent's is on its way; the daemon writes
`mcp/close` and hangs up, and the frame meets the closed socket, or the
  read side is reset first. That exited 1 with `BrokenPipeError` or
`ConnectionResetError`; a connection-level end is now exit 0 with
nothing
said, a frame past the cap still exits 1 with its reason, both pumps are
  awaited before the socket closes, and a daemon gone before the stream
opens is one sentence, not a traceback. Four deterministic cases with a
  stand-in connection fail on `c9de53947` and pass here.
- The bridge test's stand-in agent outlives its bridge, as an agent
outlives
  one of its MCP servers, so the later parts of

`test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins`
no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80
here.
- The focused set (node, MCP, remote, manager and roster files) -- 935
  passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
  simulation cases, which fail the same way on main on this computer);
  diff coverage against main 97.67% (420/430). Real machine, node
reinstalled with this tree in a directory of the check's own: a stdlib
ACP
  agent's bridge reached a host-held server and 1 MiB and 8 MiB results
  crossed whole, bridge exit 0; nothing was left on the machine.

Follow-up fix c9de53947, after the same-user token finding:

- The daemon no longer uses a token. `agent/launch` mints and writes
none,
  `mcp.token` and the `hmac` check are gone, and `Daemon.open_stream`
  identifies the bridge by the connecting process instead (`_peer_pid` +
  `Daemon.task_of`). The same-user path the reviewer kept open -- a peer
reading another task's `mcp.token` and opening its stream -- is closed,
  because identity is the connection the kernel reports, not a readable
  secret.
- Tests, each failing on the unfixed tree first: a bridge placed under
task
  B that names task A is refused (`the connection is a process of task
  'agent@box-2', not 'agent@box-1'`), as are a process under no task, an
  unplaced connection, and a task the daemon does not run
(`tests/test_node_server.py`); a real-OS test launches an agent, takes
its
real child, and asserts the child resolves to the task while the test's
own
  process resolves to none (runs on Linux in CI); `_peer_pid` is checked
against a real socketpair; the in-process and subprocess bridge tests
now
identify by the connecting process, and a bridge this test spawns
itself,
  or another task's agent spawns, never opens the stream.
- Windows withholding now covers a row on this computer too, not only a
  machine row: `session_mcp_delivered` drops the `machine and` guard and
answers no whenever `unix_sockets_available()` is false, so a local acp
row is no longer advertised as injectable and no longer raises at
dispatch
on a host without the transport; `_mcp_endpoints` yields nothing before
opening an endpoint and `_mcp_note` explains it. The two tests the
Windows
review named now guard on `unix_sockets_available()`, and the
withholding
  test is parameterized over a machine row and a local row.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over the node/MCP/remote suite -- 97.63% changed-line coverage
(412/422).
  `ruff`, `ty` on the changed modules, and the import-cycle budget pass.
- Real machine (ravenx2, this head, node reinstalled, digest
fdbe132e76f0):
a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB
and
  an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the
  connecting-process identity admits the legitimate bridge end to end on
  real Linux; the cross-task refusal on Linux is the real-OS test above,
  which CI runs there.

Earlier review fixes (4dc5b3e1f, the task-to-grant binding). Its
host-side
binding of each dispatch's endpoints to the acquired agent task is kept;
the
daemon-side token this run verified was since replaced by the
connecting-process
identity above (c9de53947), so the token-named tests and reproduction
below no
longer describe the current tree.


- `uv run --no-sync pytest tests/test_node_client.py
tests/test_node_server.py tests/test_node_bridge.py
tests/test_node_relay.py tests/test_node_bundle.py
tests/test_node_install.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py
tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict
--cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report=
--tb=short` -- 1495 passed and one minimal-node stdio response test
failed;
  strict idle accounting also flagged the assembly test. Both cases pass
together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or
timing limits were weakened. The new authorization, revocation and host
  compatibility tests all pass.
- The five focused cases failed on the unfixed implementation first: a
  different authenticated task selecting another grant, a stream with no
task, each missing Unix socket API, and an astral Unicode chunk over the
node's byte limit. Regressions also cover revocation, grant replacement
and link replacement while an endpoint connection is opening, and a real
  remote turn continuing when both host Unix socket APIs are absent.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
  -- 97.94% (381/389 changed executable lines); this fix alone is 100%
  (40/40). Python lint, type, dependency
  and import checks pass; all 10 import contracts hold. The scope checks
  for commit messages, source language and file sizes pass. Both RPC
  generators pass `--check` after integrating current main.
- A real Linux node with two concurrent tasks and a host loopback
endpoint:
  before, the attacker's own task token plus the victim's public grant
  names returns the victim endpoint's marker; after, it returns no data
  and the endpoint is never reached. Its own grant works, unknown and
revoked grants fail, and the same attack remains denied after the relay
  is killed and an ordinary ping reattaches both tasks. A full chunk of
  262144 astral characters was lost/corrupted before and arrives intact
after. The final documented source tree was reinstalled and checked
again.
- Real ACP dispatch through the node's bridge reaches a host-held stdio
MCP tool and returns its marker; its grant is removed when the turn
ends.
  With the host Unix socket APIs removed to simulate Windows, the remote
  agent still completes, receives no MCP, explains the host limitation,
  and the host MCP ledger records no extra call. This is not a native
  Windows run. Dedicated test tasks, host endpoints and bridge processes
  are cleaned up.
- CONTEXT.md, module/field docstrings, both collaboration guides, the
  external design notes and this description were checked against the
  implementation. Main's no-debug-switch behavior and full-turn recovery
  documentation are retained.

Full combined-tree verification: `uv run --no-sync pytest -n 4
--tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure
reproduces on current main in a separate checkout: host CLI color
rendering,
LibreOffice detection and the two simulation cases. No other test fails.
The minimal-node response test that failed in the focused parallel
selection
passes in this full run. Diff coverage: 97.94% (381/389 executable
changed lines).

Earlier author verification:

- `uv run pytest tests/test_node_bridge.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_bundle.py tests/test_node_install.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py`
  -- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge
  and a stdio node naming none, the token written owner-only beside the
  record, a stream opened with the task's token and refused with another
  or for a task not running, refused with no host holding the task,
  refused past 16 per task, one per bridge connection, the pieces of a
  frame crossing both ways as they are and a stranger carrying nothing,
  and every way a stream ends (the bridge closing, the host closing, the
  bridge's connection ending, the host's link ending, the agent ending)
  with the other end told; the bridge as a process against a real daemon
  over its socket: a frame to the host and one back whole, a large frame
  each way in pieces, the agent closing its stdin ending the stream and
the bridge leaving clean, the host closing the stream ending the bridge,
  a wrong token and a missing daemon each refused on stderr with exit 1;
  the stand-in machine with the real daemon and the real bridge: an acp
stub there that connects to its `mcpServers` and calls a tool reaches a
  stdio server here and answers with its result, the stanza handed to it
  naming the node's bridge, the task, the dispatch and the server and
  nothing of the server's definition or socket, the ledger here showing
  one call, no bridge outliving the dispatch; the reply's note and the
  roster predicate for a machine row.
- `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed,
4 failed:
  `test_rpc_files.py::test_a_host_without_libreoffice_says_so`,

`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`,

`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`
and
`test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`,
which fail
  the same way on main on this machine.
- Real machine (a registered Linux machine with Claude Code signed in,
  the node reinstalled with the code before and after):
  A28, before (the
  code of #934's head and its node): the turn closed with "MCP servers
  'probe' were not lent", Claude Code said no tool of that name was
  connected, and the server here saw no call; after: `PROBE_OK:r2-lend`
  in 9.4 s, the server here recorded the call, and the turn closed with
"lent to acp agent ... through the Raven node on machine ...". A34:
after
the turn the machine's process list shows no bridge, a grep of its node
  directory finds nothing of the server's command or of this computer's
  socket directory, and no endpoint socket is left here. A29: with the
tool switched off here, Claude Code answered UNAVAILABLE and the server
saw no call. A30: a 40 s tool call under way, the bridge visible in the
  machine's process list, then cancelled: 3 s later no bridge there and
  no socket here. A31: the relay ssh killed during a 20 s call: the turn
ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next
dispatch reached the server again in 7.6 s. A32: five calls in one turn
  all reached the server and came back; an 8 MiB answer: the server here
  produced it, and a plain ACP agent on the machine (a stdlib stub that
speaks the stanza's bridge directly) received it whole through the node
-- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude
  Code itself closed the server on that answer and reported the call
  failed: the agent's own cap on a tool result, not the link's.
- `ruff`, the commit-message and PR-body checks -- clean.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

The English and Chinese collaboration guides describe lending,
task-bound authorization,
Windows withholding, and the existing CLI/restart limitations.

## Risk

Registered-machine support is enabled by configuration, without the
removed
trial environment switch. An ACP entry on a machine receives its granted
MCP servers on a Unix-socket-capable host; the host's normal MCP policy
still
applies. This intentionally exposes those tools to the selected remote
agent.
The server definition, credentials and endpoint paths remain on the
host.

The daemon identifies a bridge's task by the connecting process (the
peer
pid the kernel reports, walked to the agent it started), not by any
token or
name the bridge presents, so a peer task sharing the OS account cannot
open
another task's stream; the host then authorizes that task against the
dispatch/server grant. A system that reports no peer pid opens no
stream.
On Windows hosts remote lending is withheld explicitly, without aborting
the
remote task. Protocol 6 remains the MCP wire version; host and node must
also
have the same code digest, so updated nodes must be reinstalled.
Rollback is
a revert; the machine then carries no stream and receives no bridge
stanza.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

#934 (merged); #920 and #922 (merged).

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-15 15:37:50 +08:00

558 lines
25 KiB
Python

"""raven.node.bridge: the stdio MCP server an agent on a machine spawns, carrying one stream to a host endpoint.
The bridge is run the way an agent runs it: spawned by a process the daemon
started as a task (``node_bridge_agent.py`` stands in for the agent, relaying
its stdin and stdout), on the same machine as a real daemon; the host's end is
this test, on the daemon's socket. What is checked is the carrying -- the
agent's frames reach the host in pieces and whole, the host's reach the
agent's stdout whole, the stream ends when either side does -- and whose a
bridge is: the daemon knows it by the process that connected, so one spawned
by this test, or by another task's agent, never begins. The bridge's code is
also driven in this process, with the daemon told the test is a process of
the task, for the lines the subprocess cannot show.
"""
from __future__ import annotations
import asyncio
import io
import json
import os
import shlex
import subprocess
import sys
import time
from pathlib import Path
from types import SimpleNamespace
from typing import Any
import pytest
from raven.mcp.bridge import unix_sockets_available
if sys.platform == "win32" or not unix_sockets_available():
pytest.skip("the daemon is POSIX: it locks with fcntl and listens on a unix socket", allow_module_level=True)
from raven.node import bridge, protocol, server
pytestmark = pytest.mark.slow
_AGENT = Path(__file__).with_name("node_bridge_agent.py")
def test_a_frame_is_cut_into_the_pieces_a_line_holds_and_the_last_is_marked(monkeypatch):
monkeypatch.setattr(protocol, "MCP_CHUNK", 4)
assert protocol.pieces(b'{"a":1}') == [('{"a"', False), (":1}", True)]
assert protocol.pieces(b"") == [("", True)], "an empty frame is one piece, so the end is said"
assert protocol.pieces("héllo".encode()) == [("héll", False), ("o", True)], "by characters, never inside one"
class _Host:
"""The host's end of a daemon's connection, over its socket: asks, and every notification as it comes."""
def __init__(self, reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
self.reader = reader
self.writer = writer
self.frames: list[dict[str, Any]] = []
self._collecting = asyncio.create_task(self._collect())
async def _collect(self) -> None:
while line := await self.reader.readline():
self.frames.append(json.loads(line))
async def ask(self, request_id: int, method: str, params: dict[str, Any]) -> dict[str, Any]:
self.writer.write(
json.dumps({"jsonrpc": "2.0", "id": request_id, "method": method, "params": params}).encode() + b"\n"
)
await self.writer.drain()
await self.until(lambda: any(f.get("id") == request_id for f in self.frames))
return next(f for f in self.frames if f.get("id") == request_id)
async def tell(self, method: str, params: dict[str, Any]) -> None:
self.writer.write(json.dumps({"jsonrpc": "2.0", "method": method, "params": params}).encode() + b"\n")
await self.writer.drain()
def notified(self, method: str) -> list[dict[str, Any]]:
return [f["params"] for f in self.frames if f.get("method") == method]
def lines(self, channel: int = 1) -> list[str]:
"""What the agent on ``channel`` wrote: the bridge's stdout as the agent relays it, and its last word."""
return [p["line"] for p in self.notified(protocol.AGENT_FRAME) if p.get("channel") == channel]
async def say(self, text: str, *, channel: int = 1, eof: bool = False) -> None:
"""A line to the agent's stdin, which the agent hands to its bridge; ``eof`` ends both."""
await self.tell(protocol.AGENT_STDIN, {"channel": channel, "data": text, "eof": eof})
async def until(self, check: Any, *, timeout: float = 15.0) -> None:
deadline = time.monotonic() + timeout
while not check():
assert time.monotonic() < deadline, "did not happen in time"
await asyncio.sleep(0.02)
async def close(self) -> None:
self._collecting.cancel()
self.writer.close()
await self.writer.wait_closed()
async def _daemon(root: Path, monkeypatch: pytest.MonkeyPatch) -> tuple[asyncio.Task[int], _Host]:
monkeypatch.setattr(server, "_IDLE_CHECK_S", 0.1)
running = asyncio.create_task(server.run_daemon_async(root, idle_s=0.5))
sock = server.socket_path(root)
deadline = time.monotonic() + 10
while not sock.exists() and time.monotonic() < deadline:
await asyncio.sleep(0.05)
# Lines of pieces, each up to MCP_CHUNK characters plus their JSON: the
# default 64 KiB limit would drop the collector on the first big one.
reader, writer = await asyncio.open_unix_connection(str(sock), limit=4 * 1024 * 1024)
host = _Host(reader, writer)
await host.ask(1, protocol.HELLO, {"protocol": protocol.PROTOCOL})
return running, host
async def _wind_down(host: _Host, running: asyncio.Task[int], *, channels: tuple[int, ...] = (1,)) -> None:
"""End the agents, hang up, and see the daemon leave idle; a daemon that does not is cancelled rather than waited on."""
try:
for n, channel in enumerate(channels):
await host.ask(90 + n, protocol.AGENT_CANCEL, {"channel": channel})
finally:
await host.close()
try:
assert await asyncio.wait_for(running, timeout=15) == 0, "nothing to do: the daemon leaves"
except asyncio.TimeoutError:
running.cancel()
raise
def _bridge_argv(root: Path, *, task: str = "a@box-1", lend: str = "d1", server_name: str = "pg") -> list[str]:
argv = [sys.executable, "-I", "-m", "raven.node", "--mcp", "--dir", str(root)]
return [*argv, "--task", task, "--lend", lend, "--server", server_name]
def _bridge(root: Path, **kw: Any) -> subprocess.Popen[bytes]:
"""A bridge this test spawns itself: a process of no task's, as the daemon sees it."""
return subprocess.Popen(
_bridge_argv(root, **kw), stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE
) # noqa: S603
async def _agent(host: _Host, root: Path, tmp_path: Path, *, task: str, channel: int = 1, **kw: Any) -> int:
"""An agent of ``task`` on the daemon, spawning the bridge with ``kw``'s names; its pid."""
command = shlex.join([sys.executable, str(_AGENT), *_bridge_argv(root, **kw)])
started = await host.ask(
10 + channel,
protocol.AGENT_LAUNCH,
{"channel": channel, "task": task, "command": command, "cwd": str(tmp_path)},
)
assert started["result"]["task"] == task
return started["result"]["pid"]
async def test_the_bridge_an_agent_spawns_carries_its_frames_to_the_host_and_the_host_s_back_whole(
tmp_path, monkeypatch
):
root = tmp_path / "n"
running, host = await _daemon(root, monkeypatch)
await _agent(host, root, tmp_path, task="a@box-1")
try:
# What the agent writes reaches the host as a stream of its task, in pieces that make the frame.
await host.say('{"jsonrpc":"2.0","id":1,"method":"initialize"}\n')
await host.until(lambda: host.notified(protocol.MCP_FRAME))
assert host.notified(protocol.MCP_OPEN) == [{"task": "a@box-1", "stream": 1, "server": "pg", "lend": "d1"}]
assert host.notified(protocol.MCP_FRAME) == [
{"stream": 1, "text": '{"jsonrpc":"2.0","id":1,"method":"initialize"}', "end": True}
]
# What the host sends, whole or in pieces, is one line on the agent's side.
await host.tell(protocol.MCP_FRAME, {"stream": 1, "text": '{"jsonrpc":"2.0","id":1,"result":{}}', "end": True})
big = '{"jsonrpc":"2.0","id":2,"result":"' + "x" * (protocol.MCP_CHUNK + 10) + '"}'
for text, end in protocol.pieces(big.encode()):
await host.tell(protocol.MCP_FRAME, {"stream": 1, "text": text, "end": end})
await host.until(lambda: len(host.lines()) == 2)
assert host.lines() == ['{"jsonrpc":"2.0","id":1,"result":{}}', big]
# A frame of the agent's longer than a line crosses in pieces too.
long = '{"jsonrpc":"2.0","id":3,"params":"' + "y" * (protocol.MCP_CHUNK + 5) + '"}'
await host.say(long + "\n")
await host.until(
lambda: len(host.notified(protocol.MCP_FRAME)) >= 3 and host.notified(protocol.MCP_FRAME)[-1]["end"]
)
crossed = "".join(p["text"] for p in host.notified(protocol.MCP_FRAME)[1:])
assert crossed == long and [p["end"] for p in host.notified(protocol.MCP_FRAME)[1:]] == [False, True]
# The agent closes the server's stdin: the stream is over, the host is told, the bridge leaves clean.
await host.say("", eof=True)
await host.until(lambda: host.notified(protocol.MCP_CLOSE))
assert host.notified(protocol.MCP_CLOSE) == [
{"stream": 1, "task": "a@box-1", "reason": "the agent closed the server"}
]
await host.until(lambda: host.lines()[-1:] == ["BRIDGE_EXIT 0 "])
await host.until(lambda: host.notified(protocol.AGENT_EXIT))
finally:
await _wind_down(host, running)
async def test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins(
tmp_path, monkeypatch
):
root = tmp_path / "n"
running, host = await _daemon(root, monkeypatch)
try:
await _agent(host, root, tmp_path, task="a@box-1")
await host.say('{"jsonrpc":"2.0","id":1,"method":"initialize"}\n')
await host.until(lambda: host.notified(protocol.MCP_OPEN))
await host.tell(protocol.MCP_CLOSE, {"stream": 1, "reason": "the dispatch ended"})
await host.until(lambda: host.lines())
assert host.lines() == ["BRIDGE_EXIT 0 "], (
"told the stream is over, it leaves, having written nothing of its own"
)
# The agent outlives its server, so the task the bridges below name is still running.
assert [p for p in host.notified(protocol.AGENT_EXIT) if p.get("channel") == 1] == []
# Another task's agent spawns a bridge that names this task, this dispatch and this server: the daemon
# knows whose process it is, and refuses it as that task's.
await _agent(host, root, tmp_path, task="b@box-1", channel=2)
await host.say('{"jsonrpc":"2.0","id":1,"method":"initialize"}\n', channel=2)
await host.until(lambda: host.lines(2))
(said,) = host.lines(2)
assert said == (
"BRIDGE_EXIT 1 raven node mcp bridge: the daemon refused the stream: "
"the connection is a process of task 'b@box-1', not 'a@box-1'"
)
assert host.notified(protocol.MCP_OPEN) == [{"task": "a@box-1", "stream": 1, "server": "pg", "lend": "d1"}]
# A bridge spawned by nobody's agent -- this test's own process -- is no task's.
stranger = _bridge(root)
_out, err = await asyncio.to_thread(stranger.communicate, b'{"jsonrpc":"2.0","id":1}\n', 15)
assert stranger.returncode == 1
assert err.strip() == (
b"raven node mcp bridge: the daemon refused the stream: "
b"the connection is not a process of any task on this machine"
)
nowhere = _bridge(tmp_path / "no-such-node")
_out, err = await asyncio.to_thread(nowhere.communicate, b"", 15)
assert nowhere.returncode == 1 and b"cannot reach the daemon under" in err
finally:
await _wind_down(host, running, channels=(1, 2))
# ---- the bridge run in this process, on the daemon's socket ----------------------------------------
class _Out:
"""What the bridge writes to the agent, as its stdout would take it."""
def __init__(self) -> None:
self.data = b""
def write(self, data: bytes) -> None:
self.data += data
def flush(self) -> None:
pass
def lines(self) -> list[bytes]:
return self.data.splitlines()
def _carry(root: Path, stdin: asyncio.StreamReader, out: _Out, **kw: Any) -> asyncio.Task[int]:
opts = {"task": "a@box-1", "lend": "d1", "server_name": "pg", **kw}
return asyncio.create_task(bridge.run_mcp_async(root, stdin=stdin, out=out, **opts))
def _of_task(monkeypatch: pytest.MonkeyPatch, task: str | None) -> None:
"""Make the daemon place this process (the in-process bridge's) as ``task``'s, or, with ``None``, as no task's.
Only the identity read is made up, not the process table: ``_end_tree``
reads the real one to end the agent's group at teardown, and a made-up
table that put this process under the agent would have the cancel kill
the test itself.
"""
real = server.Daemon.task_of
def placed(self: server.Daemon, pid: int) -> Any:
if pid == os.getpid():
return self.tasks.get(task) if task is not None else None
return real(self, pid)
monkeypatch.setattr(server.Daemon, "task_of", placed)
async def test_run_in_this_process_the_bridge_carries_frames_both_ways_and_leaves_when_the_agent_closes_its_stdin(
tmp_path, monkeypatch
):
root = tmp_path / "n"
running, host = await _daemon(root, monkeypatch)
started = await host.ask(
2, protocol.AGENT_LAUNCH, {"channel": 1, "task": "a@box-1", "command": "sleep 30", "cwd": str(tmp_path)}
)
_of_task(monkeypatch, "a@box-1")
stdin, out = asyncio.StreamReader(), _Out()
carrying = _carry(root, stdin, out)
try:
stdin.feed_data(b'{"jsonrpc":"2.0","id":1,"method":"initialize"}\n')
await host.until(lambda: host.notified(protocol.MCP_FRAME))
assert host.notified(protocol.MCP_OPEN) == [{"task": "a@box-1", "stream": 1, "server": "pg", "lend": "d1"}]
assert host.notified(protocol.MCP_FRAME) == [
{"stream": 1, "text": '{"jsonrpc":"2.0","id":1,"method":"initialize"}', "end": True}
]
await host.tell(protocol.MCP_FRAME, {"stream": 1, "text": '{"jsonrpc":"2.0","id":1,"result":{}}', "end": True})
big = '{"jsonrpc":"2.0","id":2,"result":"' + "x" * (protocol.MCP_CHUNK + 10) + '"}'
for text, end in protocol.pieces(big.encode()):
await host.tell(protocol.MCP_FRAME, {"stream": 1, "text": text, "end": end})
await host.until(lambda: len(out.lines()) == 2)
assert out.lines() == [b'{"jsonrpc":"2.0","id":1,"result":{}}', big.encode()], "whole lines, however they came"
long = '{"jsonrpc":"2.0","id":3,"params":"' + "y" * (protocol.MCP_CHUNK + 5) + '"}'
stdin.feed_data(long.encode() + b"\n")
await host.until(
lambda: len(host.notified(protocol.MCP_FRAME)) >= 3 and host.notified(protocol.MCP_FRAME)[-1]["end"]
)
assert "".join(p["text"] for p in host.notified(protocol.MCP_FRAME)[1:]) == long
stdin.feed_eof()
assert await asyncio.wait_for(carrying, timeout=15) == 0
await host.until(lambda: host.notified(protocol.MCP_CLOSE))
assert host.notified(protocol.MCP_CLOSE) == [
{"stream": 1, "task": "a@box-1", "reason": "the agent closed the server"}
]
finally:
if not carrying.done():
carrying.cancel()
await _wind_down(host, running)
async def test_the_bridge_says_why_it_never_began_and_why_it_stopped(tmp_path, monkeypatch, capsys):
root = tmp_path / "n"
running, host = await _daemon(root, monkeypatch)
started = await host.ask(
2, protocol.AGENT_LAUNCH, {"channel": 1, "task": "a@box-1", "command": "sleep 30", "cwd": str(tmp_path)}
)
agent = started["result"]["pid"]
try:
# No daemon to reach under that directory.
assert await _carry(tmp_path / "no-such-node", asyncio.StreamReader(), _Out()) == 1
assert "cannot reach the daemon under" in capsys.readouterr().err
# The daemon refuses: this process is no task's, as the real process table has it.
assert await _carry(root, asyncio.StreamReader(), _Out()) == 1
assert (
"the daemon refused the stream: the connection is not a process of any task on this machine"
in capsys.readouterr().err
)
# The daemon refuses: this process is another task's.
other = await host.ask(
3, protocol.AGENT_LAUNCH, {"channel": 2, "task": "b@box-1", "command": "sleep 30", "cwd": str(tmp_path)}
)
_of_task(monkeypatch, "b@box-1")
assert await _carry(root, asyncio.StreamReader(), _Out()) == 1
assert "the daemon refused the stream: the connection is a process of task 'b@box-1', not 'a@box-1'" in (
capsys.readouterr().err
)
# A frame of the agent's past the cap ends the bridge, and says so; the host hears the stream end.
_of_task(monkeypatch, "a@box-1")
monkeypatch.setattr(protocol, "MCP_MAX_FRAME", 64)
stdin = asyncio.StreamReader()
carrying = _carry(root, stdin, _Out())
await host.until(lambda: host.notified(protocol.MCP_OPEN))
stdin.feed_data(b'{"jsonrpc":"2.0","id":1,"params":"' + b"z" * 100 + b'"}\n')
assert await asyncio.wait_for(carrying, timeout=15) == 1
assert "ValueError: frame exceeds 64 bytes" in capsys.readouterr().err
await host.until(lambda: host.notified(protocol.MCP_CLOSE))
assert host.notified(protocol.MCP_CLOSE)[-1]["reason"] == "the bridge ended"
finally:
await _wind_down(host, running, channels=(1, 2))
async def test_the_bridge_reads_only_its_stream_s_frames_off_the_daemon_and_leaves_when_told_the_stream_is_over(
tmp_path,
):
"""A stand-in daemon on the socket: what the bridge makes of lines that are not its stream's, and of a close."""
openings: list[dict[str, Any]] = []
async def answering(reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
openings.append(json.loads(await reader.readline()))
for line in (
b'{"jsonrpc":"2.0","id":1,"result":{"stream":7}}\n',
b"not json\n",
b'["not an object"]\n',
b'{"jsonrpc":"2.0","method":"mcp/frame","params":{"stream":2,"text":"other","end":true}}\n',
b'{"jsonrpc":"2.0","method":"mcp/frame","params":{"stream":7,"text":5,"end":false}}\n',
b'{"jsonrpc":"2.0","method":"something/else","params":{"stream":7}}\n',
b'{"jsonrpc":"2.0","method":"mcp/frame","params":{"stream":7,"text":"a","end":false}}\n',
b'{"jsonrpc":"2.0","method":"mcp/frame","params":{"stream":7,"text":"b","end":true}}\n',
b'{"jsonrpc":"2.0","method":"mcp/close","params":{"stream":7,"reason":"the dispatch ended"}}\n',
):
writer.write(line)
await writer.drain()
await reader.read()
writer.close()
async def silent(reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
await reader.readline()
writer.close()
async def refusing(reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
await reader.readline()
writer.write(b'{"jsonrpc":"2.0","id":1,"error":{"code":-32015,"message":"no host holds task"}}\n')
await writer.drain()
writer.close()
out = _Out()
for name, handler, want in (("a", answering, 0), ("b", silent, 1), ("c", refusing, 1)):
root = tmp_path / name
server.own_dir(root / server.RUN_DIR)
daemon = await asyncio.start_unix_server(handler, path=str(server.socket_path(root)))
try:
assert await asyncio.wait_for(_carry(root, asyncio.StreamReader(), out), timeout=15) == want
finally:
daemon.close()
await daemon.wait_closed()
assert openings == [
{"jsonrpc": "2.0", "id": 1, "method": "mcp/open", "params": {"task": "a@box-1", "lend": "d1", "server": "pg"}}
]
assert out.lines() == [b"ab"], "its own stream's frame, put together; nothing of the rest"
# ---- a daemon side that goes under the stream --------------------------------------------------
class _GoneReader:
"""The bridge's read side of a daemon connection: the lines given, then the error given, else nothing more."""
def __init__(self, lines: list[bytes], then: BaseException | None = None) -> None:
self.lines = list(lines)
self.then = then
self._never = asyncio.Event()
async def readline(self) -> bytes:
if self.lines:
return self.lines.pop(0)
if self.then is not None:
raise self.then
await self._never.wait()
return b""
class _GoneWriter:
"""The bridge's write side: every line kept, and a broken pipe from the ``breaks_at``-th line's drain on."""
def __init__(self, breaks_at: int | None = None) -> None:
self.sent: list[bytes] = []
self.breaks_at = breaks_at
self.closed = False
def write(self, data: bytes) -> None:
self.sent.append(data)
async def drain(self) -> None:
if self.breaks_at is not None and len(self.sent) >= self.breaks_at:
raise BrokenPipeError(32, "Broken pipe")
def close(self) -> None:
self.closed = True
async def wait_closed(self) -> None:
return None
_OPENED = b'{"jsonrpc":"2.0","id":1,"result":{"stream":4}}\n'
def _connected(monkeypatch: pytest.MonkeyPatch, reader: _GoneReader, writer: _GoneWriter) -> None:
async def connect(path: str, *, limit: int) -> tuple[_GoneReader, _GoneWriter]:
return reader, writer
monkeypatch.setattr(bridge.asyncio, "open_unix_connection", connect)
def _pumps_left() -> list[str]:
"""The bridge's own tasks still pending: none, once it has returned."""
return [
task.get_coro().__qualname__
for task in asyncio.all_tasks()
if not task.done() and task.get_coro().__qualname__.startswith("run_mcp_async.<locals>.")
]
@pytest.mark.parametrize(
("then", "breaks_at", "agent_says"),
[
(None, 2, b'{"jsonrpc":"2.0","id":1,"method":"initialize"}\n'),
(ConnectionResetError(54, "Connection reset by peer"), None, b""),
],
ids=["a broken pipe under the agent's frame", "a reset under the daemon's"],
)
async def test_a_stream_whose_daemon_side_goes_under_it_ends_as_streams_end(
tmp_path, monkeypatch, capsys, then, breaks_at, agent_says
):
"""How a stream the host closed can end when a frame of the agent's is on its way: the daemon writes mcp/close
and hangs up, and the frame meets the closed socket, or the read side is reset before mcp/close is read. Either
way the stream is over: the bridge leaves as a stream ends, says nothing, and leaves no pump behind."""
writer = _GoneWriter(breaks_at)
_connected(monkeypatch, _GoneReader([_OPENED], then), writer)
stdin = asyncio.StreamReader()
stdin.feed_data(agent_says)
rc = await asyncio.wait_for(_carry(tmp_path / "n", stdin, _Out()), timeout=15)
assert (rc, capsys.readouterr().err) == (0, "")
assert _pumps_left() == [] and writer.closed
async def test_a_frame_past_the_cap_is_said_and_leaves_no_pump_behind(tmp_path, monkeypatch, capsys):
"""A fault of the stream's own is still said, unlike the daemon side going: a frame past the cap ends the bridge
with its reason, and the pump reading the daemon is over before the socket closes."""
monkeypatch.setattr(protocol, "MCP_MAX_FRAME", 64)
_connected(monkeypatch, _GoneReader([_OPENED]), _GoneWriter())
stdin = asyncio.StreamReader()
stdin.feed_data(b'{"jsonrpc":"2.0","id":1,"params":"' + b"z" * 100 + b'"}\n')
assert await asyncio.wait_for(_carry(tmp_path / "n", stdin, _Out()), timeout=15) == 1
assert capsys.readouterr().err == "raven node mcp bridge: ValueError: frame exceeds 64 bytes\n"
assert _pumps_left() == []
@pytest.mark.parametrize("breaks", ["on the opening", "before the answer"])
async def test_a_daemon_that_goes_before_the_stream_opens_is_one_sentence(tmp_path, monkeypatch, capsys, breaks):
reset = ConnectionResetError(54, "Connection reset by peer")
if breaks == "on the opening":
_connected(monkeypatch, _GoneReader([]), _GoneWriter(breaks_at=1))
why = "[Errno 32] Broken pipe"
else:
_connected(monkeypatch, _GoneReader([], reset), _GoneWriter())
why = "[Errno 54] Connection reset by peer"
assert await asyncio.wait_for(_carry(tmp_path / "n", asyncio.StreamReader(), _Out()), timeout=15) == 1
assert capsys.readouterr().err == (
f"raven node mcp bridge: the daemon closed the connection before the stream opened: {why}\n"
)
def test_run_mcp_takes_the_agent_s_stdin_as_a_pipe_and_keeps_its_stdout_for_frames(tmp_path, monkeypatch, capsys):
read_end, write_end = os.pipe()
frames = io.BytesIO()
with os.fdopen(read_end, "rb") as stdin:
monkeypatch.setattr(sys, "stdin", stdin)
monkeypatch.setattr(sys, "stdout", SimpleNamespace(buffer=frames))
rc = bridge.run_mcp(tmp_path / "no-such-node", task="a@box-1", lend="d1", server_name="pg")
os.close(write_end)
assert rc == 1 and "cannot reach the daemon under" in capsys.readouterr().err
assert frames.getvalue() == b"", "nothing but frames goes to the agent"
async def test_a_full_astral_piece_fits_the_daemons_byte_limited_reader():
text = chr(0x1F600) * protocol.MCP_CHUNK
wire = bridge._line(protocol.MCP_FRAME, {"stream": 1, "text": text, "end": True})
reader = asyncio.StreamReader(limit=server._LINE_LIMIT)
reader.feed_data(wire)
reader.feed_eof()
assert json.loads(await reader.readline())["params"]["text"] == text