## Summary Bundled ACP agents could fail to start on Windows when rendering configuration, reading stdin, or replacing the launcher process. Closing an ACP connection could also fail before releasing its process and transport because Windows has no os.killpg. - Keep all five bundled Windows launchers alive until their ACP child exits, forward the exit status, and apply the same behavior to new scaffolded launchers. POSIX retains exec. - Preserve live rendered configurations with the cross-platform PID probe, retain POSIX file permissions, and preserve seed-file line endings. - Read Windows stdin pipes through a daemon feeder with PeekNamedPipe and reads bounded by the available bytes. - End the live launcher's Windows process tree with bounded taskkill before releasing the transport. Leave an already-exited launcher's PID alone. - Preserve a completed session/new reply when EOF shutdown cancels its optional model-menu lookup. The first LiteLLM import can exceed the five-second grace even after #909 removed the metadata-network delay. Active-connection cancellation and cancellation during required session setup still propagate. - Run real stdio handshakes, all five launchers, command registration, and native teardown cases in Windows CI. The handshake now also runs with a fresh bytecode cache. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Native Windows build 10.0.26300, Python 3.13.12, PYTHONUTF8=1. The final tree is based on the fetched main revision; all eight patches replayed unchanged. The last upstream update adds bug-report upload/receiver code and has no file overlap with this PR. The 19-case selection below was rerun on the final tree; the broader selection and controls predate that last unrelated update. Tests used isolated user/Raven homes, frozen dependencies, and verified imports from the tested checkout. Run-owned --basetemp and --junitxml output paths are omitted from the selections below. - `uv run --frozen --no-sync pytest tests/test_acp_server.py::TestFrameLoop::test_session_creation_answers_when_eof_cancels_the_catalogue tests/test_acp_methods.py::TestConfigOptions::test_session_creation_preserves_cancellation_outside_shutdown_metadata tests/integration/test_acp_stdio_smoke.py tests/test_cli_agents_commands.py::test_register_pins_the_resolved_row_into_the_host_roster tests/test_subagent_acp.py::test_close_without_killpg_ends_the_tree_through_taskkill tests/test_subagent_acp.py::test_close_without_killpg_kills_nothing_by_the_pid_of_an_exited_launcher -m 'integration or not integration' -n 0 --tb=short`: **19 passed, no skips** after the final rebase. This includes both EOF timings, cancellation controls, both bytecode-cache states, all five real launchers, and real taskkill. One non-failing idle-ceiling warning was reported. - `uv run --frozen --no-sync pytest tests/test_acp_server.py tests/test_acp_methods.py tests/test_acp_updates.py tests/test_acp_config_options.py tests/test_cli_acp_commands.py tests/test_config_product_render.py tests/test_utils_pid.py tests/test_agents_code_launcher.py tests/test_agents_design_launcher.py tests/test_agents_oncall_launcher.py tests/test_agents_ppt_launcher.py tests/test_agents_research_launcher.py tests/test_cli_agents_commands.py tests/test_subagent_acp.py tests/test_rpc_bootstrap.py tests/test_subagent_host_env.py tests/test_subagent_probe.py -n 0 --tb=short`: **1284 passed, 58 failed, 10 skipped, 4 errors**. The same selection on the rebased PR immediately before the EOF fix returned **1280 passed, the identical 58 failing and 4 error test IDs, and 10 skipped**. No new failure/error IDs from the EOF fix. - All 62 failure/error IDs were also selected on main: 56 failed, 4 errored, and 2 passed. Those two passed in isolation on this PR too, and failed in the complete selection both before and after the EOF fix. The remaining failures include Windows path, shell/command, and symlink-privilege cases. The attempted complete main selection stalled in the pre-existing Windows stdin tests and was terminated; no complete main-suite result is claimed. - Before the EOF fix, `uv run --frozen --no-sync pytest tests/integration/test_acp_stdio_smoke.py::test_a_whole_handshake_runs_in_one_process[cold_imports] -m integration -n 0 --tb=short` failed with response IDs [1, 3]. A separate stack-capture probe measured 6.76 seconds in the first LiteLLM import; the handler awaited model.options while its worker imported LiteLLM and its dependencies. - `uv run --frozen --no-sync pytest tests/test_acp_server.py::TestFrameLoop::test_session_creation_answers_when_eof_cancels_the_catalogue tests/test_acp_methods.py::TestConfigOptions::test_session_creation_preserves_cancellation_outside_shutdown_metadata -n 0 --tb=short`: before the fix, 2 failed and 2 passed. Both EOF cases failed with the missing session reply. They pass with the fix and are included in the broader post-rebase selection above. Warm/cold real handshakes plus these four cases also passed as a six-case selection. - A mutant that treats every connection as closing makes the ordinary-cancellation control fail. Cancellation during required session setup still propagates. - `uv run --frozen --no-sync ruff check raven evolver agents plugins-dist tests scripts docs-site` and the corresponding `ruff format --check`: passed; 2266 files already formatted. - `uv run --frozen --no-sync lint-imports`: 10 contracts kept. `uv run --frozen --no-sync ty check raven/acp/methods.py raven/acp/updates.py raven/acp_client/client.py raven/cli/acp_commands.py raven/config/product_render.py`: passed. - `uv run --frozen --no-sync pre-commit run --from-ref origin/main --to-ref HEAD`: every applicable hook passed. The two final docstring edits also passed their file-scoped pre-commit run. - `node node_modules/@commitlint/cli/cli.js --from origin/main --to HEAD --config commitlint.config.cjs`, `uv run --frozen --no-sync python scripts/check_commit_messages.py origin/main..HEAD`, `uv run --frozen --no-sync python scripts/check_large_files.py origin/main...HEAD`, `uv run --frozen --no-sync python scripts/check_source_language.py origin/main...HEAD`, and `git diff --check origin/main...HEAD`: passed. - The PR body passed the repository body checker and an explicit ASCII scan. The broader Windows selection remains red, so the relevant-tests checkbox is unchecked. This revision has no new Linux or full repository run, live LLM workload, rendering workload, or ACL audit. POSIX-only skips are not counted as Windows passes. - [ ] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed ## Risk - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes At EOF, a completed session/new can omit optional configOptions when shutdown cancels a slow lookup; the sessionId and session modes are retained. Active connections keep their existing model-menu behavior. The handler grace remains five seconds. Windows keeps the launcher present while its server runs, and connection close waits up to five seconds for taskkill. Idle Windows pipe input polls every 10 ms. Redirected files and text streams retain their existing reader path. POSIX retains exec and 0600 permissions; Windows rendered files inherit directory ACLs. There is no configuration-format change. Previously generated agent folders retain their existing launchers; the scaffold change applies to new folders. Revert this PR to roll back the launcher, stdin, process-cleanup and EOF-response changes. ## Related Issues #909
834 lines
30 KiB
Python
834 lines
30 KiB
Python
"""The connection: its frame loop, its concurrency, and its teardown.
|
|
|
|
The engine is stubbed. What is under test is everything around it, and the one
|
|
claim worth the file is that a suspended ``session/prompt`` does not block the
|
|
``session/cancel`` that has to reach it -- handling frames inline would make
|
|
cancellation, the one operation the protocol requires to always work,
|
|
unreachable.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
import io
|
|
import json
|
|
|
|
import pytest
|
|
from loguru import logger
|
|
|
|
from raven.acp import server
|
|
from raven.acp.updates import UpdateTranslator
|
|
|
|
|
|
class _Broker:
|
|
"""The one method the ACP question routing calls on the runtime's broker."""
|
|
|
|
def __init__(self) -> None:
|
|
self.answers: list[tuple[str, str]] = []
|
|
|
|
def reply(self, key: str, answer: str) -> bool:
|
|
self.answers.append((key, answer))
|
|
return True
|
|
|
|
|
|
class _Stack:
|
|
"""The subset of ``RpcStack`` the server touches, with a teardown log."""
|
|
|
|
def __init__(self, *, teardown_error: Exception | None = None) -> None:
|
|
from raven.rpc.dispatcher import Dispatcher
|
|
|
|
self.dispatcher = Dispatcher()
|
|
self.agent_loop = None
|
|
self.torn_down = 0
|
|
# The runtime's ask-user broker. Present on ``RpcStack``, and the ACP
|
|
# question routing answers it -- a stub without it would let a missing
|
|
# wire-up pass.
|
|
self.question_broker = _Broker()
|
|
self._teardown_error = teardown_error
|
|
|
|
async def teardown(self) -> None:
|
|
self.torn_down += 1
|
|
if self._teardown_error is not None:
|
|
raise self._teardown_error
|
|
|
|
|
|
def _reader(payload: bytes) -> asyncio.StreamReader:
|
|
reader = asyncio.StreamReader()
|
|
reader.feed_data(payload)
|
|
reader.feed_eof()
|
|
return reader
|
|
|
|
|
|
def _frames(out: io.BytesIO) -> list[dict]:
|
|
return [json.loads(line) for line in out.getvalue().decode("utf-8").splitlines() if line]
|
|
|
|
|
|
@pytest.fixture
|
|
def stub(monkeypatch):
|
|
stack = _Stack()
|
|
|
|
async def _build(translator, *, channel=server.ACP_CHANNEL, approval_responder=None):
|
|
stack.channel = channel
|
|
stack.approval_responder = approval_responder
|
|
return stack
|
|
|
|
monkeypatch.setattr(server, "build_stack", _build)
|
|
return stack
|
|
|
|
|
|
class TestFrameLoop:
|
|
async def test_it_answers_each_request_and_stays_silent_on_notifications(self, stub):
|
|
out = io.BytesIO()
|
|
payload = (
|
|
b'{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}\n'
|
|
b'{"jsonrpc":"2.0","method":"$/cancel_request","params":{"id":1}}\n'
|
|
b'{"jsonrpc":"2.0","id":2,"method":"session/load","params":{}}\n'
|
|
)
|
|
|
|
await server.serve(_reader(payload), out)
|
|
|
|
assert [f["id"] for f in _frames(out)] == [1, 2]
|
|
|
|
async def test_an_unreadable_frame_is_reported_on_the_wire(self, stub):
|
|
"""The client is what has to learn its frame was unreadable; a log line
|
|
reaches nobody who can act on it."""
|
|
out = io.BytesIO()
|
|
|
|
await server.serve(_reader(b'garbage\n{"jsonrpc":"2.0","id":3,"method":"initialize","params":{}}\n'), out)
|
|
|
|
frames = _frames(out)
|
|
assert frames[0]["error"]["code"] == -32700
|
|
assert frames[0]["id"] is None
|
|
assert frames[1]["id"] == 3
|
|
|
|
async def test_a_client_that_says_nothing_is_answered_with_nothing(self, stub):
|
|
out = io.BytesIO()
|
|
|
|
await server.serve(_reader(b""), out)
|
|
|
|
assert out.getvalue() == b""
|
|
|
|
async def test_the_engine_is_built_on_the_acp_channel(self, stub):
|
|
"""Sharing ``"tui"`` would put an editor's sessions in the terminal's
|
|
picker, because the session listing filters on the key prefix."""
|
|
await server.serve(_reader(b""), io.BytesIO())
|
|
|
|
assert stub.channel == "acp"
|
|
assert server.ACP_CHANNEL == "acp"
|
|
|
|
async def test_session_creation_with_uncached_ollama_models_finishes_before_eof_cleanup(
|
|
self, stub, tmp_path, monkeypatch
|
|
):
|
|
from litellm.llms.ollama.completion.transformation import OllamaConfig
|
|
|
|
from raven.providers.litellm_setup import import_litellm
|
|
from raven.rpc.methods.model import model_options
|
|
|
|
model = "ollama/raven-uncached-model"
|
|
config_path = tmp_path / "config.json"
|
|
config_path.write_text(
|
|
json.dumps(
|
|
{
|
|
"agents": {"defaults": {"model": model, "provider": "ollama_chat"}},
|
|
"providers": {"ollama_chat": {"apiBase": "http://127.0.0.1:11434", "models": [model]}},
|
|
}
|
|
),
|
|
encoding="utf-8",
|
|
)
|
|
monkeypatch.setattr("raven.home._current_config_path", config_path)
|
|
import_litellm()
|
|
asked = []
|
|
|
|
def info(self, model, api_base=None):
|
|
asked.append(model)
|
|
return {}
|
|
|
|
monkeypatch.setattr(OllamaConfig, "get_model_info", info)
|
|
|
|
async def subscribe(params):
|
|
return {"subscription_id": "sub-1"}
|
|
|
|
stub.dispatcher.register("turn.subscribe", subscribe)
|
|
stub.dispatcher.register("model.options", model_options)
|
|
payload = b"".join(
|
|
json.dumps(frame).encode() + b"\n"
|
|
for frame in (
|
|
{"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {"protocolVersion": 1}},
|
|
{
|
|
"jsonrpc": "2.0",
|
|
"id": 2,
|
|
"method": "session/new",
|
|
"params": {"cwd": str(tmp_path), "mcpServers": []},
|
|
},
|
|
{
|
|
"jsonrpc": "2.0",
|
|
"id": 3,
|
|
"method": "session/prompt",
|
|
"params": {"sessionId": "acp:missing", "prompt": []},
|
|
},
|
|
)
|
|
)
|
|
out = io.BytesIO()
|
|
|
|
await server.serve(_reader(payload), out)
|
|
|
|
answers = {frame["id"]: frame for frame in _frames(out) if "id" in frame}
|
|
assert sorted(answers) == [1, 2, 3]
|
|
assert answers[2]["result"]["sessionId"].startswith("acp:")
|
|
options = answers[2]["result"]["configOptions"]
|
|
model_option = next(option for option in options if option.get("category") == "model")
|
|
assert model_option["currentValue"] == f"ollama_chat/{model}"
|
|
assert any(
|
|
option["value"] == model_option["currentValue"]
|
|
for group in model_option["options"]
|
|
for option in group["options"]
|
|
)
|
|
assert answers[3]["error"]["code"] == -32002
|
|
assert asked == []
|
|
assert stub.torn_down == 1
|
|
|
|
@pytest.mark.parametrize("eof_before_catalogue", [True, False])
|
|
async def test_session_creation_answers_when_eof_cancels_the_catalogue(
|
|
self, stub, tmp_path, monkeypatch, eof_before_catalogue
|
|
):
|
|
from tests.acp_schema import validate_def
|
|
|
|
entered = asyncio.Event()
|
|
cancelled = asyncio.Event()
|
|
unsubscribed = []
|
|
monkeypatch.setattr(server, "SHUTDOWN_GRACE_S", 0.01)
|
|
|
|
async def subscribe(params):
|
|
return {"subscription_id": "sub-1"}
|
|
|
|
async def unsubscribe(params):
|
|
unsubscribed.append(params["subscription_id"])
|
|
return {}
|
|
|
|
async def catalogue(params):
|
|
entered.set()
|
|
try:
|
|
await asyncio.Future()
|
|
finally:
|
|
cancelled.set()
|
|
|
|
stub.dispatcher.register("turn.subscribe", subscribe)
|
|
stub.dispatcher.register("turn.unsubscribe", unsubscribe)
|
|
stub.dispatcher.register("model.options", catalogue)
|
|
payload = b"".join(
|
|
json.dumps(frame).encode() + b"\n"
|
|
for frame in (
|
|
{"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {"protocolVersion": 1}},
|
|
{
|
|
"jsonrpc": "2.0",
|
|
"id": 2,
|
|
"method": "session/new",
|
|
"params": {"cwd": str(tmp_path), "mcpServers": []},
|
|
},
|
|
{
|
|
"jsonrpc": "2.0",
|
|
"id": 3,
|
|
"method": "session/prompt",
|
|
"params": {"sessionId": "acp:missing", "prompt": []},
|
|
},
|
|
)
|
|
)
|
|
reader = asyncio.StreamReader()
|
|
reader.feed_data(payload)
|
|
if eof_before_catalogue:
|
|
reader.feed_eof()
|
|
out = io.BytesIO()
|
|
serving = asyncio.create_task(server.serve(reader, out))
|
|
try:
|
|
await asyncio.wait_for(entered.wait(), timeout=2)
|
|
if not eof_before_catalogue:
|
|
reader.feed_eof()
|
|
await asyncio.wait_for(serving, timeout=2)
|
|
finally:
|
|
if not serving.done():
|
|
serving.cancel()
|
|
await asyncio.gather(serving, return_exceptions=True)
|
|
|
|
answers = {frame["id"]: frame for frame in _frames(out) if "id" in frame}
|
|
assert sorted(answers) == [1, 2, 3]
|
|
assert answers[2]["result"]["sessionId"].startswith("acp:")
|
|
assert "configOptions" not in answers[2]["result"]
|
|
validate_def("NewSessionResponse", answers[2]["result"])
|
|
assert answers[3]["error"]["code"] == -32002
|
|
assert cancelled.is_set()
|
|
assert unsubscribed == ["sub-1"]
|
|
assert stub.torn_down == 1
|
|
|
|
|
|
class TestConcurrency:
|
|
async def test_a_cancel_reaches_a_session_whose_prompt_is_still_running(self, stub, monkeypatch):
|
|
"""The claim the whole design rests on. Handled inline, the cancel would
|
|
sit behind the prompt it is meant to interrupt -- forever, since the
|
|
prompt is waiting for it."""
|
|
out = io.BytesIO()
|
|
started = asyncio.Event()
|
|
order: list[str] = []
|
|
|
|
async def _handle(frame):
|
|
method = frame.get("method")
|
|
if method == "slow":
|
|
order.append("slow-start")
|
|
started.set()
|
|
await asyncio.sleep(0.05)
|
|
order.append("slow-end")
|
|
return {"jsonrpc": "2.0", "id": frame["id"], "result": {}}
|
|
order.append(method)
|
|
return None
|
|
|
|
class _Methods:
|
|
handle = staticmethod(_handle)
|
|
|
|
monkeypatch.setattr(server, "AcpMethods", lambda **kw: _Methods())
|
|
|
|
await server.serve(
|
|
_reader(b'{"jsonrpc":"2.0","id":1,"method":"slow"}\n{"jsonrpc":"2.0","method":"fast"}\n'), out
|
|
)
|
|
|
|
assert order == ["slow-start", "fast", "slow-end"], (
|
|
"the second frame must be handled while the first is still suspended"
|
|
)
|
|
assert started.is_set()
|
|
|
|
async def test_a_finite_input_still_gets_its_answers(self, stub):
|
|
"""EOF arrives before the handlers have run at all when the input is a
|
|
fixed payload. Cancelling on EOF would answer a batch of three requests
|
|
with nothing -- which is what ``echo '...' | raven acp`` and every smoke
|
|
test look like."""
|
|
out = io.BytesIO()
|
|
payload = b"".join(b'{"jsonrpc":"2.0","id":%d,"method":"initialize","params":{}}\n' % n for n in (1, 2, 3))
|
|
|
|
await server.serve(_reader(payload), out)
|
|
|
|
assert [f["id"] for f in _frames(out)] == [1, 2, 3]
|
|
|
|
async def test_a_stuck_handler_is_cancelled_after_the_grace_period(self, stub, monkeypatch):
|
|
"""The backstop. A handler waiting on something the event stream cannot
|
|
resolve would otherwise keep the process alive after the window closed."""
|
|
out = io.BytesIO()
|
|
events: list[str] = []
|
|
monkeypatch.setattr(server, "SHUTDOWN_GRACE_S", 0.01)
|
|
|
|
async def _handle(frame):
|
|
try:
|
|
await asyncio.sleep(30)
|
|
except asyncio.CancelledError:
|
|
events.append("handler-unwound")
|
|
raise
|
|
return None
|
|
|
|
class _Methods:
|
|
handle = staticmethod(_handle)
|
|
|
|
monkeypatch.setattr(server, "AcpMethods", lambda **kw: _Methods())
|
|
original = stub.teardown
|
|
|
|
async def _teardown():
|
|
events.append("teardown")
|
|
await original()
|
|
|
|
stub.teardown = _teardown
|
|
|
|
await server.serve(_reader(b'{"jsonrpc":"2.0","id":1,"method":"slow"}\n'), out)
|
|
|
|
assert events == ["handler-unwound", "teardown"], (
|
|
"the handler must unwind before the engine it is using is torn down"
|
|
)
|
|
|
|
async def test_a_suspended_prompt_is_settled_rather_than_cancelled(self, stub, monkeypatch):
|
|
"""The difference that matters: settled, the handler returns through its
|
|
own code and writes the ``cancelled`` stop reason. Cancelled, the client
|
|
gets nothing for a request it is still holding."""
|
|
from raven.acp.updates import AcpSession
|
|
|
|
out = io.BytesIO()
|
|
captured = {}
|
|
|
|
class _Methods:
|
|
def __init__(self, **kw):
|
|
captured["translator"] = kw["translator"]
|
|
|
|
async def handle(self, frame):
|
|
translator = captured["translator"]
|
|
translator.add(AcpSession(session_id="acp:s", session_key="acp:s", cwd="/w", subscription_id="sub"))
|
|
stop = await translator.begin_turn("acp:s")
|
|
return {"jsonrpc": "2.0", "id": frame["id"], "result": {"stopReason": stop}}
|
|
|
|
monkeypatch.setattr(server, "AcpMethods", _Methods)
|
|
|
|
await server.serve(_reader(b'{"jsonrpc":"2.0","id":1,"method":"session/prompt"}\n'), out)
|
|
|
|
assert _frames(out) == [{"jsonrpc": "2.0", "id": 1, "result": {"stopReason": "cancelled"}}]
|
|
|
|
async def test_a_handler_that_finished_is_not_cancelled_again(self, stub):
|
|
out = io.BytesIO()
|
|
|
|
await server.serve(_reader(b'{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}\n'), out)
|
|
|
|
assert [f["id"] for f in _frames(out)] == [1]
|
|
|
|
|
|
class TestHandlerFailures:
|
|
async def test_a_handler_failure_is_logged_rather_than_swallowed(self, stub, monkeypatch):
|
|
"""A task's exception is retrieved by the shutdown gather and then
|
|
discarded, so a write that failed mid-session would leave no trace
|
|
anywhere."""
|
|
records = []
|
|
sink_id = logger.add(lambda message: records.append(message.record["message"]), level="ERROR")
|
|
|
|
class _Methods:
|
|
@staticmethod
|
|
async def handle(frame):
|
|
raise OSError("broken pipe")
|
|
|
|
monkeypatch.setattr(server, "AcpMethods", lambda **kw: _Methods())
|
|
try:
|
|
await server.serve(_reader(b'{"jsonrpc":"2.0","id":1,"method":"initialize"}\n'), io.BytesIO())
|
|
finally:
|
|
logger.remove(sink_id)
|
|
|
|
assert any("answering initialize failed" in message for message in records)
|
|
|
|
async def test_a_failing_handler_does_not_stop_the_loop(self, stub, monkeypatch):
|
|
seen = []
|
|
|
|
class _Methods:
|
|
@staticmethod
|
|
async def handle(frame):
|
|
seen.append(frame["id"])
|
|
raise OSError("nope")
|
|
|
|
monkeypatch.setattr(server, "AcpMethods", lambda **kw: _Methods())
|
|
|
|
await server.serve(
|
|
_reader(b'{"jsonrpc":"2.0","id":1,"method":"a"}\n{"jsonrpc":"2.0","id":2,"method":"b"}\n'),
|
|
io.BytesIO(),
|
|
)
|
|
|
|
assert seen == [1, 2]
|
|
|
|
async def test_subscriptions_are_closed_before_the_engine_is_torn_down(self, stub, monkeypatch):
|
|
order = []
|
|
|
|
class _Methods:
|
|
def __init__(self, **kw):
|
|
pass
|
|
|
|
async def handle(self, frame):
|
|
return None
|
|
|
|
async def unsubscribe_all(self):
|
|
order.append("unsubscribe")
|
|
|
|
monkeypatch.setattr(server, "AcpMethods", _Methods)
|
|
original = stub.teardown
|
|
|
|
async def _teardown():
|
|
order.append("teardown")
|
|
await original()
|
|
|
|
stub.teardown = _teardown
|
|
|
|
await server.serve(_reader(b""), io.BytesIO())
|
|
|
|
assert order == ["unsubscribe", "teardown"]
|
|
|
|
async def test_a_failing_unsubscribe_does_not_block_the_teardown(self, stub, monkeypatch):
|
|
class _Methods:
|
|
def __init__(self, **kw):
|
|
pass
|
|
|
|
async def handle(self, frame):
|
|
return None
|
|
|
|
async def unsubscribe_all(self):
|
|
raise RuntimeError("emitter gone")
|
|
|
|
monkeypatch.setattr(server, "AcpMethods", _Methods)
|
|
|
|
await server.serve(_reader(b""), io.BytesIO())
|
|
|
|
assert stub.torn_down == 1
|
|
|
|
|
|
class TestTeardown:
|
|
async def test_the_engine_is_torn_down_on_a_clean_exit(self, stub):
|
|
await server.serve(_reader(b""), io.BytesIO())
|
|
|
|
assert stub.torn_down == 1
|
|
|
|
async def test_the_engine_is_torn_down_when_the_reader_fails(self, stub):
|
|
"""A pipe that dies mid-frame must not leave the browser profile locked
|
|
and MCP subprocesses orphaned."""
|
|
reader = asyncio.StreamReader()
|
|
reader.set_exception(ConnectionResetError("pipe died"))
|
|
|
|
with pytest.raises(ConnectionResetError):
|
|
await server.serve(reader, io.BytesIO())
|
|
|
|
assert stub.torn_down == 1
|
|
|
|
async def test_a_failing_teardown_does_not_replace_a_clean_exit(self, monkeypatch):
|
|
"""Every step inside the real teardown is already guarded; this catches a
|
|
failure in the callable itself, whose traceback the client cannot see
|
|
anyway."""
|
|
stack = _Stack(teardown_error=RuntimeError("cron would not stop"))
|
|
|
|
async def _build(translator, *, channel=server.ACP_CHANNEL, approval_responder=None):
|
|
return stack
|
|
|
|
monkeypatch.setattr(server, "build_stack", _build)
|
|
|
|
await server.serve(_reader(b""), io.BytesIO())
|
|
|
|
assert stack.torn_down == 1
|
|
|
|
|
|
class TestBuildStack:
|
|
async def test_it_hands_the_translator_sink_to_the_rpc_stack(self, monkeypatch):
|
|
"""The seam that makes one sink intercept the entire outbound surface: the
|
|
emitter, all three brokers and the MCP bridge are given the same
|
|
callable."""
|
|
seen = {}
|
|
|
|
async def _build_rpc_stack(send_frame, *, channel="tui", approval_responder=None):
|
|
seen["send_frame"] = send_frame
|
|
seen["channel"] = channel
|
|
seen["approval_responder"] = approval_responder
|
|
return _Stack()
|
|
|
|
monkeypatch.setattr("raven.rpc.bootstrap.build_rpc_stack", _build_rpc_stack)
|
|
translator = UpdateTranslator(emit=lambda f: None)
|
|
|
|
await server.build_stack(translator, channel="acp", approval_responder="broker")
|
|
|
|
assert seen["send_frame"] == translator.send_frame
|
|
assert seen["channel"] == "acp"
|
|
assert seen["approval_responder"] == "broker", (
|
|
"the shell approval transport has to be replaced, or an ACP client is asked "
|
|
"for permission over a method it does not implement"
|
|
)
|
|
|
|
|
|
class TestApprovalWiring:
|
|
"""The engine's permission gate has to be told what to ask about.
|
|
|
|
Without this, an editor's agent runs ``git push``, ``npm install`` and
|
|
``curl -o`` with nothing on screen -- the built-in rulings ask about deletion
|
|
and nothing else. It is the single most visible difference between an agent
|
|
somebody trusts and one they do not.
|
|
"""
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _restore_process_families(self):
|
|
"""The declaration is process-wide, so a test that sets it has to put it
|
|
back or every later test inherits an ACP surface's policy."""
|
|
from raven.permissions import shell_policy
|
|
|
|
before = shell_policy.surface_approval_families()
|
|
yield
|
|
shell_policy.set_surface_approval_families(before)
|
|
|
|
def test_the_families_reach_a_gate_built_after_the_declaration(self):
|
|
from raven.contracts.permissions import NeedsApproval
|
|
from raven.permissions.builtin import BuiltinRulings
|
|
|
|
server._ask_before_external_effects()
|
|
rulings = BuiltinRulings()
|
|
|
|
assert isinstance(rulings.ruling("exec", {"command": "git push origin main"}), NeedsApproval)
|
|
assert isinstance(rulings.ruling("exec", {"command": "npm install lodash"}), NeedsApproval)
|
|
assert rulings.ruling("exec", {"command": "pytest -q"}) is None
|
|
|
|
def test_a_sub_agents_own_gate_inherits_them_too(self):
|
|
"""The hole this closed: a sub-agent builds its own gate with its own
|
|
rulings, so a per-instance registration reached the main loop only and a
|
|
delegated ``git push`` ran unannounced while the identical command asked
|
|
in the main agent. Declared per process, both carry it.
|
|
"""
|
|
from raven.contracts.permissions import NeedsApproval
|
|
from raven.permissions.builtin import BuiltinRulings
|
|
|
|
server._ask_before_external_effects()
|
|
main_rulings = BuiltinRulings()
|
|
sub_rulings = BuiltinRulings()
|
|
|
|
assert isinstance(main_rulings.ruling("exec", {"command": "git push origin main"}), NeedsApproval)
|
|
assert isinstance(sub_rulings.ruling("exec", {"command": "git push origin main"}), NeedsApproval)
|
|
|
|
async def test_a_delegated_command_asks_through_the_turn_it_runs_in(self):
|
|
"""A sub-agent's gate reads the turn's mode and asks through the
|
|
responder the parent turn bound (a ContextVar its task inherits). With
|
|
nobody bound it fails closed with a reason; with the editor's responder
|
|
bound, the same push reaches the human and runs on their click.
|
|
"""
|
|
from raven.config.schema import PermissionsConfig
|
|
from raven.contracts.permissions import ApprovalChoice, ApprovalOutcome
|
|
from raven.permissions.builtin import BuiltinRulings
|
|
from raven.permissions.gate import PermissionGate
|
|
from raven.permissions.turn import start_permission_turn
|
|
|
|
server._ask_before_external_effects()
|
|
gate = PermissionGate(
|
|
config_source=lambda: PermissionsConfig(mode="ask"),
|
|
builtin=BuiltinRulings(),
|
|
allow_ask=True,
|
|
)
|
|
|
|
start_permission_turn(None, conversation_id="acp", turn_id="t1")
|
|
refused = await gate.enforce("exec", {"command": "git push origin main"})
|
|
assert refused is not None
|
|
assert "not interactive" in str(refused.model_text)
|
|
|
|
class _Click:
|
|
def __init__(self) -> None:
|
|
self.seen: list[str] = []
|
|
|
|
async def await_approval(self, **request):
|
|
self.seen.append(request["description"])
|
|
return ApprovalOutcome(choice=ApprovalChoice.ALLOW)
|
|
|
|
click = _Click()
|
|
start_permission_turn(click, conversation_id="acp", turn_id="t2")
|
|
assert await gate.enforce("exec", {"command": "git push origin main"}) is None
|
|
assert click.seen == ["Publish or push work to a remote"]
|
|
|
|
def test_a_tool_built_before_the_declaration_keeps_what_it_was_born_with(self):
|
|
"""Why ``serve`` declares before it builds the engine. A policy reads the
|
|
process's families once, at construction, so a declaration that lands
|
|
afterwards reaches nothing -- and the surface would look configured while
|
|
every existing tool stayed silent.
|
|
"""
|
|
from raven.contracts.permissions import NeedsApproval
|
|
from raven.permissions.builtin import BuiltinRulings
|
|
|
|
early = BuiltinRulings()
|
|
server._ask_before_external_effects()
|
|
|
|
assert early.ruling("exec", {"command": "git push origin main"}) is None
|
|
assert isinstance(BuiltinRulings().ruling("exec", {"command": "git push origin main"}), NeedsApproval)
|
|
|
|
async def test_the_permission_broker_is_the_stacks_approval_transport(self, stub):
|
|
from raven.acp.permissions import AcpPermissionBroker
|
|
|
|
await server.serve(_reader(b""), io.BytesIO())
|
|
|
|
assert isinstance(stub.approval_responder, AcpPermissionBroker), (
|
|
"the RPC broker emits approval.request, which an ACP client does not implement"
|
|
)
|
|
|
|
|
|
class TestOutboundLifecycle:
|
|
async def test_outstanding_requests_are_failed_before_the_handlers_drain(self, stub, monkeypatch):
|
|
"""A handler suspended on a permission prompt is waiting on an outbound
|
|
future. Draining first would wait out the grace period for a promise
|
|
nothing can keep."""
|
|
from raven.acp.outbound import ConnectionClosedError
|
|
|
|
outcome = {}
|
|
|
|
class _Methods:
|
|
def __init__(self, **kw):
|
|
self.outbound = kw["outbound"]
|
|
|
|
async def handle(self, frame):
|
|
try:
|
|
await self.outbound.call("session/request_permission", {}, timeout=30)
|
|
except ConnectionClosedError:
|
|
outcome["failed"] = True
|
|
return None
|
|
|
|
async def unsubscribe_all(self):
|
|
return None
|
|
|
|
monkeypatch.setattr(server, "AcpMethods", _Methods)
|
|
|
|
await server.serve(_reader(b'{"jsonrpc":"2.0","id":1,"method":"session/prompt"}\n'), io.BytesIO())
|
|
|
|
assert outcome.get("failed") is True
|
|
|
|
async def test_a_response_frame_resolves_an_outstanding_request(self, stub):
|
|
"""The half of the loop that makes a prompt answerable: a frame with an id
|
|
and no method is not garbage, it is the answer."""
|
|
out = io.BytesIO()
|
|
answered = {}
|
|
|
|
payload = (
|
|
b'{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":1}}\n'
|
|
b'{"jsonrpc":"2.0","id":1,"result":{"outcome":{"outcome":"cancelled"}}}\n'
|
|
)
|
|
|
|
real_methods = server.AcpMethods
|
|
|
|
def _spy(**kw):
|
|
answered["outbound"] = kw["outbound"]
|
|
return real_methods(**kw)
|
|
|
|
import unittest.mock
|
|
|
|
with unittest.mock.patch.object(server, "AcpMethods", _spy):
|
|
await server.serve(_reader(payload), out)
|
|
|
|
assert answered["outbound"] is not None
|
|
|
|
|
|
class TestQuestionWiring:
|
|
async def test_a_clarify_request_reaches_the_question_routing(self, stub, monkeypatch):
|
|
"""The frame the runtime emits when ``ask_user`` fires. Dropped, it stalls
|
|
a tool call for the whole of the broker's timeout, and a client shows a
|
|
spinner and then a reply that ignores what it asked."""
|
|
taken = []
|
|
|
|
class _Questions:
|
|
def __init__(self, **kw):
|
|
pass
|
|
|
|
def set_broker(self, broker):
|
|
return None
|
|
|
|
def set_client(self, client):
|
|
return None
|
|
|
|
def handle(self, method, params):
|
|
taken.append((method, params))
|
|
return True
|
|
|
|
async def drain(self):
|
|
return None
|
|
|
|
monkeypatch.setattr(server, "AcpQuestions", _Questions)
|
|
captured = {}
|
|
real = server.UpdateTranslator
|
|
|
|
def _spy(emit, **kwargs):
|
|
captured["translator"] = real(emit, **kwargs)
|
|
return captured["translator"]
|
|
|
|
monkeypatch.setattr(server, "UpdateTranslator", _spy)
|
|
|
|
async def _serve_and_emit():
|
|
await server.serve(_reader(b""), io.BytesIO())
|
|
|
|
await _serve_and_emit()
|
|
await captured["translator"].send_frame(
|
|
{"jsonrpc": "2.0", "method": "clarify.request", "params": {"question": "which?"}}
|
|
)
|
|
|
|
assert taken == [("clarify.request", {"question": "which?"})]
|
|
|
|
async def test_questions_are_drained_before_the_outbound_futures_are_failed(self, stub, monkeypatch):
|
|
"""A round trip that has just been answered gets to deliver that answer to
|
|
the broker instead of being cancelled one step short."""
|
|
order = []
|
|
|
|
class _Questions:
|
|
def __init__(self, **kw):
|
|
pass
|
|
|
|
def set_broker(self, broker):
|
|
return None
|
|
|
|
def set_client(self, client):
|
|
return None
|
|
|
|
def handle(self, method, params):
|
|
return False
|
|
|
|
async def drain(self):
|
|
order.append("drain")
|
|
|
|
monkeypatch.setattr(server, "AcpQuestions", _Questions)
|
|
original = stub.teardown
|
|
|
|
async def _teardown():
|
|
order.append("teardown")
|
|
await original()
|
|
|
|
stub.teardown = _teardown
|
|
|
|
await server.serve(_reader(b""), io.BytesIO())
|
|
|
|
assert order == ["drain", "teardown"]
|
|
|
|
async def test_a_failing_drain_does_not_block_the_shutdown(self, stub, monkeypatch):
|
|
class _Questions:
|
|
def __init__(self, **kw):
|
|
pass
|
|
|
|
def set_broker(self, broker):
|
|
return None
|
|
|
|
def set_client(self, client):
|
|
return None
|
|
|
|
def handle(self, method, params):
|
|
return False
|
|
|
|
async def drain(self):
|
|
raise RuntimeError("a question round trip is wedged")
|
|
|
|
monkeypatch.setattr(server, "AcpQuestions", _Questions)
|
|
|
|
await server.serve(_reader(b""), io.BytesIO())
|
|
|
|
assert stub.torn_down == 1
|
|
|
|
|
|
class TestCrashHandlers:
|
|
def test_the_previous_excepthook_still_runs(self):
|
|
"""stderr is deliberately kept as a destination: an ACP client surfaces
|
|
it, and a crash that is only in a log file is a crash nobody is told
|
|
about."""
|
|
import sys
|
|
|
|
called = []
|
|
original = sys.excepthook
|
|
sys.excepthook = lambda *args: called.append(args)
|
|
try:
|
|
server.install_crash_handlers()
|
|
sys.excepthook(ValueError, ValueError("boom"), None)
|
|
finally:
|
|
sys.excepthook = original
|
|
|
|
assert called and called[0][0] is ValueError
|
|
|
|
async def test_an_asyncio_error_is_logged_with_its_exception(self):
|
|
# ``sys.excepthook`` is restored too, not only the loop handler: it is
|
|
# process-global, so leaving it installed would chain another copy onto
|
|
# every later test in the session.
|
|
import sys
|
|
|
|
records = []
|
|
sink_id = logger.add(lambda message: records.append(message.record), level="ERROR")
|
|
original_hook = sys.excepthook
|
|
try:
|
|
server.install_crash_handlers()
|
|
handler = asyncio.get_running_loop().get_exception_handler()
|
|
assert handler is not None
|
|
handler(asyncio.get_running_loop(), {"message": "task blew up", "exception": ValueError("why")})
|
|
handler(asyncio.get_running_loop(), {"message": "no exception here", "future": "x"})
|
|
finally:
|
|
logger.remove(sink_id)
|
|
asyncio.get_running_loop().set_exception_handler(None)
|
|
sys.excepthook = original_hook
|
|
|
|
assert any("task blew up" in r["message"] for r in records)
|
|
assert any("no exception here" in r["message"] for r in records)
|
|
|
|
def test_it_installs_the_hook_even_with_no_loop_running(self):
|
|
"""The excepthook half is what covers a failure during startup, before
|
|
the loop exists."""
|
|
import sys
|
|
|
|
original = sys.excepthook
|
|
try:
|
|
server.install_crash_handlers()
|
|
|
|
assert sys.excepthook is not original
|
|
finally:
|
|
sys.excepthook = original
|