## Summary Adds MCP lending for remote ACP agents on top of the machine and recovery services already merged in #920, #922 and #934. The branch includes current main and its default-on registered-machine behavior. What this does: an acp entry on a registered machine is handed the MCP servers its row grants, the way an entry on this computer is, and the servers themselves stay here with their definitions and keys. - The agent on the machine runs the node's own bridge as a stdio MCP server (`python -I -m raven.node --mcp --dir <node dir> --task <task> --lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte pumps around a connection to the daemon's socket, no MCP parsed, no credential held, the standard library only. - The daemon carries the bridge's frames as a *stream* to the host holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6, `raven/node/protocol.py`, `raven/node/server.py`). The daemon tells which task a bridge is by the process that connected -- the peer pid off the socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the parent chain to the agent the daemon started, which the bridge is under and no other task's process is; the kernel supplies that pid and a peer cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the task it names must be the one the connection is a process of. Two tasks on one machine run as one user, so no name or file a bridge could present would tell them apart; the connecting process does. One stream per bridge, at most 16 per task; a frame travels in pieces of at most 256 KiB characters, the last marked (`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the daemon relays and keeps nothing. A stream ends with either end, the host's link or the task, and the other end is told. - The host opens the endpoints for a machine row exactly as for a row here (`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in the host's sandbox, HTTP auth added here), binds them to the machine, dispatch and acquired agent task (`NodePool.lend`/`unlend`) before handing out bridge stanzas. On `mcp/open` the host checks the daemon-authenticated task against that binding before connecting; it rechecks the grant and link after the socket opens. An unlend, replacement grant or replaced link cannot authorize that pending connection. The host then pumps the frames both ways (`raven/node/client.py`). `node/hello` names the bridge; a node serving without a daemon (`--stdio`) names none, and the dispatch then goes without MCP and says so. - The reply closes with the servers lent and the machine's node, or with why none were. Remote lending requires Unix-socket support on the host: on Windows it is withheld before endpoint setup, the task continues, and the reply explains why. The roster's "injectable MCP" matches that support (`session_mcp_delivered`). The cli lane on a machine still withholds its servers (the next change); the openai lane has no tools. Not in this change: the cli lane's MCP file on the machine; a stream that survives the link going (a bridge sees its server end, the agent reports the call failed, and the next dispatch works again); MCP for a run picked up after this computer restarted (#934): it reports its calls fail. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Follow-up fix 05eaba67a, after the c9de53947 review: - The nine tests the Windows review named now pin the host fact through one fixture, `unix_socket_host` (`tests/conftest.py`), which sets `raven.mcp.bridge.unix_sockets_available` to true and re-points every `raven.*` module that imported it by name; their bodies are unchanged. - Found complete, not just the nine: the whole suite was run with `asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted before collection (a pytest plugin, controller and xdist workers), at the merge base `6b938a426` and at `c9de53947`, and the two failure sets compared. 17 node ids failed at the head and not at the base: the nine, and eight this change added that run the transport for real (the bridge module, and two lending tests through the stand-in machine). Those eight are guarded by host capability instead of pinned: the bridge module skips when `sys.platform == "win32" or not unix_sockets_available()`, the two lending tests carry the existing `skipif(not unix_sockets_available())`. After the fix the same run fails the same 66 node ids at the head as at the base: nothing this change adds or touches fails on a host without unix sockets. - A bridge whose daemon side goes ends as streams end. The host can close a stream while a frame of the agent's is on its way; the daemon writes `mcp/close` and hangs up, and the frame meets the closed socket, or the read side is reset first. That exited 1 with `BrokenPipeError` or `ConnectionResetError`; a connection-level end is now exit 0 with nothing said, a frame past the cap still exits 1 with its reason, both pumps are awaited before the socket closes, and a daemon gone before the stream opens is one sentence, not a traceback. Four deterministic cases with a stand-in connection fail on `c9de53947` and pass here. - The bridge test's stand-in agent outlives its bridge, as an agent outlives one of its MCP servers, so the later parts of `test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins` no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80 here. - The focused set (node, MCP, remote, manager and roster files) -- 935 passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed (`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the two simulation cases, which fail the same way on main on this computer); diff coverage against main 97.67% (420/430). Real machine, node reinstalled with this tree in a directory of the check's own: a stdlib ACP agent's bridge reached a host-held server and 1 MiB and 8 MiB results crossed whole, bridge exit 0; nothing was left on the machine. Follow-up fix c9de53947, after the same-user token finding: - The daemon no longer uses a token. `agent/launch` mints and writes none, `mcp.token` and the `hmac` check are gone, and `Daemon.open_stream` identifies the bridge by the connecting process instead (`_peer_pid` + `Daemon.task_of`). The same-user path the reviewer kept open -- a peer reading another task's `mcp.token` and opening its stream -- is closed, because identity is the connection the kernel reports, not a readable secret. - Tests, each failing on the unfixed tree first: a bridge placed under task B that names task A is refused (`the connection is a process of task 'agent@box-2', not 'agent@box-1'`), as are a process under no task, an unplaced connection, and a task the daemon does not run (`tests/test_node_server.py`); a real-OS test launches an agent, takes its real child, and asserts the child resolves to the task while the test's own process resolves to none (runs on Linux in CI); `_peer_pid` is checked against a real socketpair; the in-process and subprocess bridge tests now identify by the connecting process, and a bridge this test spawns itself, or another task's agent spawns, never opens the stream. - Windows withholding now covers a row on this computer too, not only a machine row: `session_mcp_delivered` drops the `machine and` guard and answers no whenever `unix_sockets_available()` is false, so a local acp row is no longer advertised as injectable and no longer raises at dispatch on a host without the transport; `_mcp_endpoints` yields nothing before opening an endpoint and `_mcp_note` explains it. The two tests the Windows review named now guard on `unix_sockets_available()`, and the withholding test is parameterized over a machine row and a local row. - `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` over the node/MCP/remote suite -- 97.63% changed-line coverage (412/422). `ruff`, `ty` on the changed modules, and the import-cycle budget pass. - Real machine (ravenx2, this head, node reinstalled, digest fdbe132e76f0): a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB and an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the connecting-process identity admits the legitimate bridge end to end on real Linux; the cross-task refusal on Linux is the real-OS test above, which CI runs there. Earlier review fixes (4dc5b3e1f, the task-to-grant binding). Its host-side binding of each dispatch's endpoints to the acquired agent task is kept; the daemon-side token this run verified was since replaced by the connecting-process identity above (c9de53947), so the token-named tests and reproduction below no longer describe the current tree. - `uv run --no-sync pytest tests/test_node_client.py tests/test_node_server.py tests/test_node_bridge.py tests/test_node_relay.py tests/test_node_bundle.py tests/test_node_install.py tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py tests/test_acp_remote.py tests/test_remote_file_tools.py tests/test_subagent_probe.py tests/test_subagent_acp.py tests/test_import_cycle_budget.py tests/test_resume_remote.py tests/test_remote_resume.py tests/test_subagent_manager.py tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict --cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report= --tb=short` -- 1495 passed and one minimal-node stdio response test failed; strict idle accounting also flagged the assembly test. Both cases pass together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or timing limits were weakened. The new authorization, revocation and host compatibility tests all pass. - The five focused cases failed on the unfixed implementation first: a different authenticated task selecting another grant, a stream with no task, each missing Unix socket API, and an astral Unicode chunk over the node's byte limit. Regressions also cover revocation, grant replacement and link replacement while an endpoint connection is opening, and a real remote turn continuing when both host Unix socket APIs are absent. - `uv run --no-sync python scripts/coverage_gate.py --coverage-json <coverage-json> diff --base-ref origin/main --threshold 90` -- 97.94% (381/389 changed executable lines); this fix alone is 100% (40/40). Python lint, type, dependency and import checks pass; all 10 import contracts hold. The scope checks for commit messages, source language and file sizes pass. Both RPC generators pass `--check` after integrating current main. - A real Linux node with two concurrent tasks and a host loopback endpoint: before, the attacker's own task token plus the victim's public grant names returns the victim endpoint's marker; after, it returns no data and the endpoint is never reached. Its own grant works, unknown and revoked grants fail, and the same attack remains denied after the relay is killed and an ordinary ping reattaches both tasks. A full chunk of 262144 astral characters was lost/corrupted before and arrives intact after. The final documented source tree was reinstalled and checked again. - Real ACP dispatch through the node's bridge reaches a host-held stdio MCP tool and returns its marker; its grant is removed when the turn ends. With the host Unix socket APIs removed to simulate Windows, the remote agent still completes, receives no MCP, explains the host limitation, and the host MCP ledger records no extra call. This is not a native Windows run. Dedicated test tasks, host endpoints and bridge processes are cleaned up. - CONTEXT.md, module/field docstrings, both collaboration guides, the external design notes and this description were checked against the implementation. Main's no-debug-switch behavior and full-turn recovery documentation are retained. Full combined-tree verification: `uv run --no-sync pytest -n 4 --tb=short --cov=raven --cov-branch --cov-report=json:<full-coverage-json> --cov-report=` returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure reproduces on current main in a separate checkout: host CLI color rendering, LibreOffice detection and the two simulation cases. No other test fails. The minimal-node response test that failed in the focused parallel selection passes in this full run. Diff coverage: 97.94% (381/389 executable changed lines). Earlier author verification: - `uv run pytest tests/test_node_bridge.py tests/test_node_server.py tests/test_node_client.py tests/test_node_relay.py tests/test_node_bundle.py tests/test_node_install.py tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py tests/test_acp_remote.py tests/test_remote_file_tools.py tests/test_subagent_probe.py tests/test_subagent_acp.py tests/test_import_cycle_budget.py tests/test_resume_remote.py tests/test_remote_resume.py tests/test_subagent_manager.py` -- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge and a stdio node naming none, the token written owner-only beside the record, a stream opened with the task's token and refused with another or for a task not running, refused with no host holding the task, refused past 16 per task, one per bridge connection, the pieces of a frame crossing both ways as they are and a stranger carrying nothing, and every way a stream ends (the bridge closing, the host closing, the bridge's connection ending, the host's link ending, the agent ending) with the other end told; the bridge as a process against a real daemon over its socket: a frame to the host and one back whole, a large frame each way in pieces, the agent closing its stdin ending the stream and the bridge leaving clean, the host closing the stream ending the bridge, a wrong token and a missing daemon each refused on stderr with exit 1; the stand-in machine with the real daemon and the real bridge: an acp stub there that connects to its `mcpServers` and calls a tool reaches a stdio server here and answers with its result, the stanza handed to it naming the node's bridge, the task, the dispatch and the server and nothing of the server's definition or socket, the ledger here showing one call, no bridge outliving the dispatch; the reply's note and the roster predicate for a machine row. - `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed, 4 failed: `test_rpc_files.py::test_a_host_without_libreoffice_says_so`, `test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`, `test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence` and `test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`, which fail the same way on main on this machine. - Real machine (a registered Linux machine with Claude Code signed in, the node reinstalled with the code before and after): A28, before (the code of #934's head and its node): the turn closed with "MCP servers 'probe' were not lent", Claude Code said no tool of that name was connected, and the server here saw no call; after: `PROBE_OK:r2-lend` in 9.4 s, the server here recorded the call, and the turn closed with "lent to acp agent ... through the Raven node on machine ...". A34: after the turn the machine's process list shows no bridge, a grep of its node directory finds nothing of the server's command or of this computer's socket directory, and no endpoint socket is left here. A29: with the tool switched off here, Claude Code answered UNAVAILABLE and the server saw no call. A30: a 40 s tool call under way, the bridge visible in the machine's process list, then cancelled: 3 s later no bridge there and no socket here. A31: the relay ssh killed during a 20 s call: the turn ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next dispatch reached the server again in 7.6 s. A32: five calls in one turn all reached the server and came back; an 8 MiB answer: the server here produced it, and a plain ACP agent on the machine (a stdlib stub that speaks the stanza's bridge directly) received it whole through the node -- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude Code itself closed the server on that answer and reported the call failed: the agent's own cap on a tool result, not the link's. - `ruff`, the commit-message and PR-body checks -- clean. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed The English and Chinese collaboration guides describe lending, task-bound authorization, Windows withholding, and the existing CLI/restart limitations. ## Risk Registered-machine support is enabled by configuration, without the removed trial environment switch. An ACP entry on a machine receives its granted MCP servers on a Unix-socket-capable host; the host's normal MCP policy still applies. This intentionally exposes those tools to the selected remote agent. The server definition, credentials and endpoint paths remain on the host. The daemon identifies a bridge's task by the connecting process (the peer pid the kernel reports, walked to the agent it started), not by any token or name the bridge presents, so a peer task sharing the OS account cannot open another task's stream; the host then authorizes that task against the dispatch/server grant. A system that reports no peer pid opens no stream. On Windows hosts remote lending is withheld explicitly, without aborting the remote task. Protocol 6 remains the MCP wire version; host and node must also have the same code digest, so updated nodes must be reinstalled. Rollback is a revert; the machine then carries no stream and receives no bridge stanza. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues #934 (merged); #920 and #922 (merged). --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com> Co-authored-by: Codex <noreply@openai.com>
558 lines
25 KiB
Python
558 lines
25 KiB
Python
"""raven.node.bridge: the stdio MCP server an agent on a machine spawns, carrying one stream to a host endpoint.
|
|
|
|
The bridge is run the way an agent runs it: spawned by a process the daemon
|
|
started as a task (``node_bridge_agent.py`` stands in for the agent, relaying
|
|
its stdin and stdout), on the same machine as a real daemon; the host's end is
|
|
this test, on the daemon's socket. What is checked is the carrying -- the
|
|
agent's frames reach the host in pieces and whole, the host's reach the
|
|
agent's stdout whole, the stream ends when either side does -- and whose a
|
|
bridge is: the daemon knows it by the process that connected, so one spawned
|
|
by this test, or by another task's agent, never begins. The bridge's code is
|
|
also driven in this process, with the daemon told the test is a process of
|
|
the task, for the lines the subprocess cannot show.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
import io
|
|
import json
|
|
import os
|
|
import shlex
|
|
import subprocess
|
|
import sys
|
|
import time
|
|
from pathlib import Path
|
|
from types import SimpleNamespace
|
|
from typing import Any
|
|
|
|
import pytest
|
|
|
|
from raven.mcp.bridge import unix_sockets_available
|
|
|
|
if sys.platform == "win32" or not unix_sockets_available():
|
|
pytest.skip("the daemon is POSIX: it locks with fcntl and listens on a unix socket", allow_module_level=True)
|
|
|
|
from raven.node import bridge, protocol, server
|
|
|
|
pytestmark = pytest.mark.slow
|
|
|
|
_AGENT = Path(__file__).with_name("node_bridge_agent.py")
|
|
|
|
|
|
def test_a_frame_is_cut_into_the_pieces_a_line_holds_and_the_last_is_marked(monkeypatch):
|
|
monkeypatch.setattr(protocol, "MCP_CHUNK", 4)
|
|
|
|
assert protocol.pieces(b'{"a":1}') == [('{"a"', False), (":1}", True)]
|
|
assert protocol.pieces(b"") == [("", True)], "an empty frame is one piece, so the end is said"
|
|
assert protocol.pieces("héllo".encode()) == [("héll", False), ("o", True)], "by characters, never inside one"
|
|
|
|
|
|
class _Host:
|
|
"""The host's end of a daemon's connection, over its socket: asks, and every notification as it comes."""
|
|
|
|
def __init__(self, reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
|
|
self.reader = reader
|
|
self.writer = writer
|
|
self.frames: list[dict[str, Any]] = []
|
|
self._collecting = asyncio.create_task(self._collect())
|
|
|
|
async def _collect(self) -> None:
|
|
while line := await self.reader.readline():
|
|
self.frames.append(json.loads(line))
|
|
|
|
async def ask(self, request_id: int, method: str, params: dict[str, Any]) -> dict[str, Any]:
|
|
self.writer.write(
|
|
json.dumps({"jsonrpc": "2.0", "id": request_id, "method": method, "params": params}).encode() + b"\n"
|
|
)
|
|
await self.writer.drain()
|
|
await self.until(lambda: any(f.get("id") == request_id for f in self.frames))
|
|
return next(f for f in self.frames if f.get("id") == request_id)
|
|
|
|
async def tell(self, method: str, params: dict[str, Any]) -> None:
|
|
self.writer.write(json.dumps({"jsonrpc": "2.0", "method": method, "params": params}).encode() + b"\n")
|
|
await self.writer.drain()
|
|
|
|
def notified(self, method: str) -> list[dict[str, Any]]:
|
|
return [f["params"] for f in self.frames if f.get("method") == method]
|
|
|
|
def lines(self, channel: int = 1) -> list[str]:
|
|
"""What the agent on ``channel`` wrote: the bridge's stdout as the agent relays it, and its last word."""
|
|
return [p["line"] for p in self.notified(protocol.AGENT_FRAME) if p.get("channel") == channel]
|
|
|
|
async def say(self, text: str, *, channel: int = 1, eof: bool = False) -> None:
|
|
"""A line to the agent's stdin, which the agent hands to its bridge; ``eof`` ends both."""
|
|
await self.tell(protocol.AGENT_STDIN, {"channel": channel, "data": text, "eof": eof})
|
|
|
|
async def until(self, check: Any, *, timeout: float = 15.0) -> None:
|
|
deadline = time.monotonic() + timeout
|
|
while not check():
|
|
assert time.monotonic() < deadline, "did not happen in time"
|
|
await asyncio.sleep(0.02)
|
|
|
|
async def close(self) -> None:
|
|
self._collecting.cancel()
|
|
self.writer.close()
|
|
await self.writer.wait_closed()
|
|
|
|
|
|
async def _daemon(root: Path, monkeypatch: pytest.MonkeyPatch) -> tuple[asyncio.Task[int], _Host]:
|
|
monkeypatch.setattr(server, "_IDLE_CHECK_S", 0.1)
|
|
running = asyncio.create_task(server.run_daemon_async(root, idle_s=0.5))
|
|
sock = server.socket_path(root)
|
|
deadline = time.monotonic() + 10
|
|
while not sock.exists() and time.monotonic() < deadline:
|
|
await asyncio.sleep(0.05)
|
|
# Lines of pieces, each up to MCP_CHUNK characters plus their JSON: the
|
|
# default 64 KiB limit would drop the collector on the first big one.
|
|
reader, writer = await asyncio.open_unix_connection(str(sock), limit=4 * 1024 * 1024)
|
|
host = _Host(reader, writer)
|
|
await host.ask(1, protocol.HELLO, {"protocol": protocol.PROTOCOL})
|
|
return running, host
|
|
|
|
|
|
async def _wind_down(host: _Host, running: asyncio.Task[int], *, channels: tuple[int, ...] = (1,)) -> None:
|
|
"""End the agents, hang up, and see the daemon leave idle; a daemon that does not is cancelled rather than waited on."""
|
|
try:
|
|
for n, channel in enumerate(channels):
|
|
await host.ask(90 + n, protocol.AGENT_CANCEL, {"channel": channel})
|
|
finally:
|
|
await host.close()
|
|
try:
|
|
assert await asyncio.wait_for(running, timeout=15) == 0, "nothing to do: the daemon leaves"
|
|
except asyncio.TimeoutError:
|
|
running.cancel()
|
|
raise
|
|
|
|
|
|
def _bridge_argv(root: Path, *, task: str = "a@box-1", lend: str = "d1", server_name: str = "pg") -> list[str]:
|
|
argv = [sys.executable, "-I", "-m", "raven.node", "--mcp", "--dir", str(root)]
|
|
return [*argv, "--task", task, "--lend", lend, "--server", server_name]
|
|
|
|
|
|
def _bridge(root: Path, **kw: Any) -> subprocess.Popen[bytes]:
|
|
"""A bridge this test spawns itself: a process of no task's, as the daemon sees it."""
|
|
return subprocess.Popen(
|
|
_bridge_argv(root, **kw), stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE
|
|
) # noqa: S603
|
|
|
|
|
|
async def _agent(host: _Host, root: Path, tmp_path: Path, *, task: str, channel: int = 1, **kw: Any) -> int:
|
|
"""An agent of ``task`` on the daemon, spawning the bridge with ``kw``'s names; its pid."""
|
|
command = shlex.join([sys.executable, str(_AGENT), *_bridge_argv(root, **kw)])
|
|
started = await host.ask(
|
|
10 + channel,
|
|
protocol.AGENT_LAUNCH,
|
|
{"channel": channel, "task": task, "command": command, "cwd": str(tmp_path)},
|
|
)
|
|
assert started["result"]["task"] == task
|
|
return started["result"]["pid"]
|
|
|
|
|
|
async def test_the_bridge_an_agent_spawns_carries_its_frames_to_the_host_and_the_host_s_back_whole(
|
|
tmp_path, monkeypatch
|
|
):
|
|
root = tmp_path / "n"
|
|
running, host = await _daemon(root, monkeypatch)
|
|
await _agent(host, root, tmp_path, task="a@box-1")
|
|
try:
|
|
# What the agent writes reaches the host as a stream of its task, in pieces that make the frame.
|
|
await host.say('{"jsonrpc":"2.0","id":1,"method":"initialize"}\n')
|
|
await host.until(lambda: host.notified(protocol.MCP_FRAME))
|
|
assert host.notified(protocol.MCP_OPEN) == [{"task": "a@box-1", "stream": 1, "server": "pg", "lend": "d1"}]
|
|
assert host.notified(protocol.MCP_FRAME) == [
|
|
{"stream": 1, "text": '{"jsonrpc":"2.0","id":1,"method":"initialize"}', "end": True}
|
|
]
|
|
|
|
# What the host sends, whole or in pieces, is one line on the agent's side.
|
|
await host.tell(protocol.MCP_FRAME, {"stream": 1, "text": '{"jsonrpc":"2.0","id":1,"result":{}}', "end": True})
|
|
big = '{"jsonrpc":"2.0","id":2,"result":"' + "x" * (protocol.MCP_CHUNK + 10) + '"}'
|
|
for text, end in protocol.pieces(big.encode()):
|
|
await host.tell(protocol.MCP_FRAME, {"stream": 1, "text": text, "end": end})
|
|
await host.until(lambda: len(host.lines()) == 2)
|
|
assert host.lines() == ['{"jsonrpc":"2.0","id":1,"result":{}}', big]
|
|
|
|
# A frame of the agent's longer than a line crosses in pieces too.
|
|
long = '{"jsonrpc":"2.0","id":3,"params":"' + "y" * (protocol.MCP_CHUNK + 5) + '"}'
|
|
await host.say(long + "\n")
|
|
await host.until(
|
|
lambda: len(host.notified(protocol.MCP_FRAME)) >= 3 and host.notified(protocol.MCP_FRAME)[-1]["end"]
|
|
)
|
|
crossed = "".join(p["text"] for p in host.notified(protocol.MCP_FRAME)[1:])
|
|
assert crossed == long and [p["end"] for p in host.notified(protocol.MCP_FRAME)[1:]] == [False, True]
|
|
|
|
# The agent closes the server's stdin: the stream is over, the host is told, the bridge leaves clean.
|
|
await host.say("", eof=True)
|
|
await host.until(lambda: host.notified(protocol.MCP_CLOSE))
|
|
assert host.notified(protocol.MCP_CLOSE) == [
|
|
{"stream": 1, "task": "a@box-1", "reason": "the agent closed the server"}
|
|
]
|
|
await host.until(lambda: host.lines()[-1:] == ["BRIDGE_EXIT 0 "])
|
|
await host.until(lambda: host.notified(protocol.AGENT_EXIT))
|
|
finally:
|
|
await _wind_down(host, running)
|
|
|
|
|
|
async def test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins(
|
|
tmp_path, monkeypatch
|
|
):
|
|
root = tmp_path / "n"
|
|
running, host = await _daemon(root, monkeypatch)
|
|
try:
|
|
await _agent(host, root, tmp_path, task="a@box-1")
|
|
await host.say('{"jsonrpc":"2.0","id":1,"method":"initialize"}\n')
|
|
await host.until(lambda: host.notified(protocol.MCP_OPEN))
|
|
await host.tell(protocol.MCP_CLOSE, {"stream": 1, "reason": "the dispatch ended"})
|
|
await host.until(lambda: host.lines())
|
|
assert host.lines() == ["BRIDGE_EXIT 0 "], (
|
|
"told the stream is over, it leaves, having written nothing of its own"
|
|
)
|
|
# The agent outlives its server, so the task the bridges below name is still running.
|
|
assert [p for p in host.notified(protocol.AGENT_EXIT) if p.get("channel") == 1] == []
|
|
|
|
# Another task's agent spawns a bridge that names this task, this dispatch and this server: the daemon
|
|
# knows whose process it is, and refuses it as that task's.
|
|
await _agent(host, root, tmp_path, task="b@box-1", channel=2)
|
|
await host.say('{"jsonrpc":"2.0","id":1,"method":"initialize"}\n', channel=2)
|
|
await host.until(lambda: host.lines(2))
|
|
(said,) = host.lines(2)
|
|
assert said == (
|
|
"BRIDGE_EXIT 1 raven node mcp bridge: the daemon refused the stream: "
|
|
"the connection is a process of task 'b@box-1', not 'a@box-1'"
|
|
)
|
|
assert host.notified(protocol.MCP_OPEN) == [{"task": "a@box-1", "stream": 1, "server": "pg", "lend": "d1"}]
|
|
|
|
# A bridge spawned by nobody's agent -- this test's own process -- is no task's.
|
|
stranger = _bridge(root)
|
|
_out, err = await asyncio.to_thread(stranger.communicate, b'{"jsonrpc":"2.0","id":1}\n', 15)
|
|
assert stranger.returncode == 1
|
|
assert err.strip() == (
|
|
b"raven node mcp bridge: the daemon refused the stream: "
|
|
b"the connection is not a process of any task on this machine"
|
|
)
|
|
|
|
nowhere = _bridge(tmp_path / "no-such-node")
|
|
_out, err = await asyncio.to_thread(nowhere.communicate, b"", 15)
|
|
assert nowhere.returncode == 1 and b"cannot reach the daemon under" in err
|
|
finally:
|
|
await _wind_down(host, running, channels=(1, 2))
|
|
|
|
|
|
# ---- the bridge run in this process, on the daemon's socket ----------------------------------------
|
|
|
|
|
|
class _Out:
|
|
"""What the bridge writes to the agent, as its stdout would take it."""
|
|
|
|
def __init__(self) -> None:
|
|
self.data = b""
|
|
|
|
def write(self, data: bytes) -> None:
|
|
self.data += data
|
|
|
|
def flush(self) -> None:
|
|
pass
|
|
|
|
def lines(self) -> list[bytes]:
|
|
return self.data.splitlines()
|
|
|
|
|
|
def _carry(root: Path, stdin: asyncio.StreamReader, out: _Out, **kw: Any) -> asyncio.Task[int]:
|
|
opts = {"task": "a@box-1", "lend": "d1", "server_name": "pg", **kw}
|
|
return asyncio.create_task(bridge.run_mcp_async(root, stdin=stdin, out=out, **opts))
|
|
|
|
|
|
def _of_task(monkeypatch: pytest.MonkeyPatch, task: str | None) -> None:
|
|
"""Make the daemon place this process (the in-process bridge's) as ``task``'s, or, with ``None``, as no task's.
|
|
|
|
Only the identity read is made up, not the process table: ``_end_tree``
|
|
reads the real one to end the agent's group at teardown, and a made-up
|
|
table that put this process under the agent would have the cancel kill
|
|
the test itself.
|
|
"""
|
|
real = server.Daemon.task_of
|
|
|
|
def placed(self: server.Daemon, pid: int) -> Any:
|
|
if pid == os.getpid():
|
|
return self.tasks.get(task) if task is not None else None
|
|
return real(self, pid)
|
|
|
|
monkeypatch.setattr(server.Daemon, "task_of", placed)
|
|
|
|
|
|
async def test_run_in_this_process_the_bridge_carries_frames_both_ways_and_leaves_when_the_agent_closes_its_stdin(
|
|
tmp_path, monkeypatch
|
|
):
|
|
root = tmp_path / "n"
|
|
running, host = await _daemon(root, monkeypatch)
|
|
started = await host.ask(
|
|
2, protocol.AGENT_LAUNCH, {"channel": 1, "task": "a@box-1", "command": "sleep 30", "cwd": str(tmp_path)}
|
|
)
|
|
_of_task(monkeypatch, "a@box-1")
|
|
stdin, out = asyncio.StreamReader(), _Out()
|
|
carrying = _carry(root, stdin, out)
|
|
try:
|
|
stdin.feed_data(b'{"jsonrpc":"2.0","id":1,"method":"initialize"}\n')
|
|
await host.until(lambda: host.notified(protocol.MCP_FRAME))
|
|
assert host.notified(protocol.MCP_OPEN) == [{"task": "a@box-1", "stream": 1, "server": "pg", "lend": "d1"}]
|
|
assert host.notified(protocol.MCP_FRAME) == [
|
|
{"stream": 1, "text": '{"jsonrpc":"2.0","id":1,"method":"initialize"}', "end": True}
|
|
]
|
|
|
|
await host.tell(protocol.MCP_FRAME, {"stream": 1, "text": '{"jsonrpc":"2.0","id":1,"result":{}}', "end": True})
|
|
big = '{"jsonrpc":"2.0","id":2,"result":"' + "x" * (protocol.MCP_CHUNK + 10) + '"}'
|
|
for text, end in protocol.pieces(big.encode()):
|
|
await host.tell(protocol.MCP_FRAME, {"stream": 1, "text": text, "end": end})
|
|
await host.until(lambda: len(out.lines()) == 2)
|
|
assert out.lines() == [b'{"jsonrpc":"2.0","id":1,"result":{}}', big.encode()], "whole lines, however they came"
|
|
|
|
long = '{"jsonrpc":"2.0","id":3,"params":"' + "y" * (protocol.MCP_CHUNK + 5) + '"}'
|
|
stdin.feed_data(long.encode() + b"\n")
|
|
await host.until(
|
|
lambda: len(host.notified(protocol.MCP_FRAME)) >= 3 and host.notified(protocol.MCP_FRAME)[-1]["end"]
|
|
)
|
|
assert "".join(p["text"] for p in host.notified(protocol.MCP_FRAME)[1:]) == long
|
|
|
|
stdin.feed_eof()
|
|
assert await asyncio.wait_for(carrying, timeout=15) == 0
|
|
await host.until(lambda: host.notified(protocol.MCP_CLOSE))
|
|
assert host.notified(protocol.MCP_CLOSE) == [
|
|
{"stream": 1, "task": "a@box-1", "reason": "the agent closed the server"}
|
|
]
|
|
finally:
|
|
if not carrying.done():
|
|
carrying.cancel()
|
|
await _wind_down(host, running)
|
|
|
|
|
|
async def test_the_bridge_says_why_it_never_began_and_why_it_stopped(tmp_path, monkeypatch, capsys):
|
|
root = tmp_path / "n"
|
|
running, host = await _daemon(root, monkeypatch)
|
|
started = await host.ask(
|
|
2, protocol.AGENT_LAUNCH, {"channel": 1, "task": "a@box-1", "command": "sleep 30", "cwd": str(tmp_path)}
|
|
)
|
|
agent = started["result"]["pid"]
|
|
try:
|
|
# No daemon to reach under that directory.
|
|
assert await _carry(tmp_path / "no-such-node", asyncio.StreamReader(), _Out()) == 1
|
|
assert "cannot reach the daemon under" in capsys.readouterr().err
|
|
|
|
# The daemon refuses: this process is no task's, as the real process table has it.
|
|
assert await _carry(root, asyncio.StreamReader(), _Out()) == 1
|
|
assert (
|
|
"the daemon refused the stream: the connection is not a process of any task on this machine"
|
|
in capsys.readouterr().err
|
|
)
|
|
|
|
# The daemon refuses: this process is another task's.
|
|
other = await host.ask(
|
|
3, protocol.AGENT_LAUNCH, {"channel": 2, "task": "b@box-1", "command": "sleep 30", "cwd": str(tmp_path)}
|
|
)
|
|
_of_task(monkeypatch, "b@box-1")
|
|
assert await _carry(root, asyncio.StreamReader(), _Out()) == 1
|
|
assert "the daemon refused the stream: the connection is a process of task 'b@box-1', not 'a@box-1'" in (
|
|
capsys.readouterr().err
|
|
)
|
|
|
|
# A frame of the agent's past the cap ends the bridge, and says so; the host hears the stream end.
|
|
_of_task(monkeypatch, "a@box-1")
|
|
monkeypatch.setattr(protocol, "MCP_MAX_FRAME", 64)
|
|
stdin = asyncio.StreamReader()
|
|
carrying = _carry(root, stdin, _Out())
|
|
await host.until(lambda: host.notified(protocol.MCP_OPEN))
|
|
stdin.feed_data(b'{"jsonrpc":"2.0","id":1,"params":"' + b"z" * 100 + b'"}\n')
|
|
assert await asyncio.wait_for(carrying, timeout=15) == 1
|
|
assert "ValueError: frame exceeds 64 bytes" in capsys.readouterr().err
|
|
await host.until(lambda: host.notified(protocol.MCP_CLOSE))
|
|
assert host.notified(protocol.MCP_CLOSE)[-1]["reason"] == "the bridge ended"
|
|
finally:
|
|
await _wind_down(host, running, channels=(1, 2))
|
|
|
|
|
|
async def test_the_bridge_reads_only_its_stream_s_frames_off_the_daemon_and_leaves_when_told_the_stream_is_over(
|
|
tmp_path,
|
|
):
|
|
"""A stand-in daemon on the socket: what the bridge makes of lines that are not its stream's, and of a close."""
|
|
openings: list[dict[str, Any]] = []
|
|
|
|
async def answering(reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
|
|
openings.append(json.loads(await reader.readline()))
|
|
for line in (
|
|
b'{"jsonrpc":"2.0","id":1,"result":{"stream":7}}\n',
|
|
b"not json\n",
|
|
b'["not an object"]\n',
|
|
b'{"jsonrpc":"2.0","method":"mcp/frame","params":{"stream":2,"text":"other","end":true}}\n',
|
|
b'{"jsonrpc":"2.0","method":"mcp/frame","params":{"stream":7,"text":5,"end":false}}\n',
|
|
b'{"jsonrpc":"2.0","method":"something/else","params":{"stream":7}}\n',
|
|
b'{"jsonrpc":"2.0","method":"mcp/frame","params":{"stream":7,"text":"a","end":false}}\n',
|
|
b'{"jsonrpc":"2.0","method":"mcp/frame","params":{"stream":7,"text":"b","end":true}}\n',
|
|
b'{"jsonrpc":"2.0","method":"mcp/close","params":{"stream":7,"reason":"the dispatch ended"}}\n',
|
|
):
|
|
writer.write(line)
|
|
await writer.drain()
|
|
await reader.read()
|
|
writer.close()
|
|
|
|
async def silent(reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
|
|
await reader.readline()
|
|
writer.close()
|
|
|
|
async def refusing(reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
|
|
await reader.readline()
|
|
writer.write(b'{"jsonrpc":"2.0","id":1,"error":{"code":-32015,"message":"no host holds task"}}\n')
|
|
await writer.drain()
|
|
writer.close()
|
|
|
|
out = _Out()
|
|
for name, handler, want in (("a", answering, 0), ("b", silent, 1), ("c", refusing, 1)):
|
|
root = tmp_path / name
|
|
server.own_dir(root / server.RUN_DIR)
|
|
daemon = await asyncio.start_unix_server(handler, path=str(server.socket_path(root)))
|
|
try:
|
|
assert await asyncio.wait_for(_carry(root, asyncio.StreamReader(), out), timeout=15) == want
|
|
finally:
|
|
daemon.close()
|
|
await daemon.wait_closed()
|
|
|
|
assert openings == [
|
|
{"jsonrpc": "2.0", "id": 1, "method": "mcp/open", "params": {"task": "a@box-1", "lend": "d1", "server": "pg"}}
|
|
]
|
|
assert out.lines() == [b"ab"], "its own stream's frame, put together; nothing of the rest"
|
|
|
|
|
|
# ---- a daemon side that goes under the stream --------------------------------------------------
|
|
|
|
|
|
class _GoneReader:
|
|
"""The bridge's read side of a daemon connection: the lines given, then the error given, else nothing more."""
|
|
|
|
def __init__(self, lines: list[bytes], then: BaseException | None = None) -> None:
|
|
self.lines = list(lines)
|
|
self.then = then
|
|
self._never = asyncio.Event()
|
|
|
|
async def readline(self) -> bytes:
|
|
if self.lines:
|
|
return self.lines.pop(0)
|
|
if self.then is not None:
|
|
raise self.then
|
|
await self._never.wait()
|
|
return b""
|
|
|
|
|
|
class _GoneWriter:
|
|
"""The bridge's write side: every line kept, and a broken pipe from the ``breaks_at``-th line's drain on."""
|
|
|
|
def __init__(self, breaks_at: int | None = None) -> None:
|
|
self.sent: list[bytes] = []
|
|
self.breaks_at = breaks_at
|
|
self.closed = False
|
|
|
|
def write(self, data: bytes) -> None:
|
|
self.sent.append(data)
|
|
|
|
async def drain(self) -> None:
|
|
if self.breaks_at is not None and len(self.sent) >= self.breaks_at:
|
|
raise BrokenPipeError(32, "Broken pipe")
|
|
|
|
def close(self) -> None:
|
|
self.closed = True
|
|
|
|
async def wait_closed(self) -> None:
|
|
return None
|
|
|
|
|
|
_OPENED = b'{"jsonrpc":"2.0","id":1,"result":{"stream":4}}\n'
|
|
|
|
|
|
def _connected(monkeypatch: pytest.MonkeyPatch, reader: _GoneReader, writer: _GoneWriter) -> None:
|
|
async def connect(path: str, *, limit: int) -> tuple[_GoneReader, _GoneWriter]:
|
|
return reader, writer
|
|
|
|
monkeypatch.setattr(bridge.asyncio, "open_unix_connection", connect)
|
|
|
|
|
|
def _pumps_left() -> list[str]:
|
|
"""The bridge's own tasks still pending: none, once it has returned."""
|
|
return [
|
|
task.get_coro().__qualname__
|
|
for task in asyncio.all_tasks()
|
|
if not task.done() and task.get_coro().__qualname__.startswith("run_mcp_async.<locals>.")
|
|
]
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
("then", "breaks_at", "agent_says"),
|
|
[
|
|
(None, 2, b'{"jsonrpc":"2.0","id":1,"method":"initialize"}\n'),
|
|
(ConnectionResetError(54, "Connection reset by peer"), None, b""),
|
|
],
|
|
ids=["a broken pipe under the agent's frame", "a reset under the daemon's"],
|
|
)
|
|
async def test_a_stream_whose_daemon_side_goes_under_it_ends_as_streams_end(
|
|
tmp_path, monkeypatch, capsys, then, breaks_at, agent_says
|
|
):
|
|
"""How a stream the host closed can end when a frame of the agent's is on its way: the daemon writes mcp/close
|
|
and hangs up, and the frame meets the closed socket, or the read side is reset before mcp/close is read. Either
|
|
way the stream is over: the bridge leaves as a stream ends, says nothing, and leaves no pump behind."""
|
|
writer = _GoneWriter(breaks_at)
|
|
_connected(monkeypatch, _GoneReader([_OPENED], then), writer)
|
|
stdin = asyncio.StreamReader()
|
|
stdin.feed_data(agent_says)
|
|
|
|
rc = await asyncio.wait_for(_carry(tmp_path / "n", stdin, _Out()), timeout=15)
|
|
|
|
assert (rc, capsys.readouterr().err) == (0, "")
|
|
assert _pumps_left() == [] and writer.closed
|
|
|
|
|
|
async def test_a_frame_past_the_cap_is_said_and_leaves_no_pump_behind(tmp_path, monkeypatch, capsys):
|
|
"""A fault of the stream's own is still said, unlike the daemon side going: a frame past the cap ends the bridge
|
|
with its reason, and the pump reading the daemon is over before the socket closes."""
|
|
monkeypatch.setattr(protocol, "MCP_MAX_FRAME", 64)
|
|
_connected(monkeypatch, _GoneReader([_OPENED]), _GoneWriter())
|
|
stdin = asyncio.StreamReader()
|
|
stdin.feed_data(b'{"jsonrpc":"2.0","id":1,"params":"' + b"z" * 100 + b'"}\n')
|
|
|
|
assert await asyncio.wait_for(_carry(tmp_path / "n", stdin, _Out()), timeout=15) == 1
|
|
assert capsys.readouterr().err == "raven node mcp bridge: ValueError: frame exceeds 64 bytes\n"
|
|
assert _pumps_left() == []
|
|
|
|
|
|
@pytest.mark.parametrize("breaks", ["on the opening", "before the answer"])
|
|
async def test_a_daemon_that_goes_before_the_stream_opens_is_one_sentence(tmp_path, monkeypatch, capsys, breaks):
|
|
reset = ConnectionResetError(54, "Connection reset by peer")
|
|
if breaks == "on the opening":
|
|
_connected(monkeypatch, _GoneReader([]), _GoneWriter(breaks_at=1))
|
|
why = "[Errno 32] Broken pipe"
|
|
else:
|
|
_connected(monkeypatch, _GoneReader([], reset), _GoneWriter())
|
|
why = "[Errno 54] Connection reset by peer"
|
|
|
|
assert await asyncio.wait_for(_carry(tmp_path / "n", asyncio.StreamReader(), _Out()), timeout=15) == 1
|
|
assert capsys.readouterr().err == (
|
|
f"raven node mcp bridge: the daemon closed the connection before the stream opened: {why}\n"
|
|
)
|
|
|
|
|
|
def test_run_mcp_takes_the_agent_s_stdin_as_a_pipe_and_keeps_its_stdout_for_frames(tmp_path, monkeypatch, capsys):
|
|
read_end, write_end = os.pipe()
|
|
frames = io.BytesIO()
|
|
with os.fdopen(read_end, "rb") as stdin:
|
|
monkeypatch.setattr(sys, "stdin", stdin)
|
|
monkeypatch.setattr(sys, "stdout", SimpleNamespace(buffer=frames))
|
|
rc = bridge.run_mcp(tmp_path / "no-such-node", task="a@box-1", lend="d1", server_name="pg")
|
|
os.close(write_end)
|
|
|
|
assert rc == 1 and "cannot reach the daemon under" in capsys.readouterr().err
|
|
assert frames.getvalue() == b"", "nothing but frames goes to the agent"
|
|
|
|
|
|
async def test_a_full_astral_piece_fits_the_daemons_byte_limited_reader():
|
|
text = chr(0x1F600) * protocol.MCP_CHUNK
|
|
wire = bridge._line(protocol.MCP_FRAME, {"stream": 1, "text": text, "end": True})
|
|
reader = asyncio.StreamReader(limit=server._LINE_LIMIT)
|
|
reader.feed_data(wire)
|
|
reader.feed_eof()
|
|
assert json.loads(await reader.readline())["params"]["text"] == text
|