Raven-1/tests/test_tool_refusal_continuation.py
Dizhan Xue 547c1aba5e
feat(*): reach Mirasim's agents as ACP sub-agent presets (#971)
## Summary

Connect the coding agents Mirasim runs as Raven ACP sub-agents, with one
bundled preset per agent and no separate adapter download.

- Serve Claude Code, Codex, Kimi Code, Antigravity, Qwen Code, Grok
  Build, DSH, Pi and ZCode CLI through `mirasim ui-cli`. The adapter
  runs on Raven's interpreter, so a Mirasim preset is not offered on a
  registered machine, and a stored row whose interpreter is gone is
  pointed at the running one when the config is next read. It withholds
  session MCP because ui-cli does not accept per-session servers. The
  boot capability backfill asks Mirasim once which agents it has
  installed and handshakes only those.
- Read model and effort choices from Mirasim. Offer the own models of
  Claude Code, Codex, Kimi Code and ZCode CLI on both their own-account
  and Mirasim platform routes. Persist conversation bindings and switch
  routes before continued turns. A menu read that fails is tried again
  after a short pause rather than kept for the cached copy's lifetime,
  an effort the menus in hand do not offer is read again before it is
  refused, and a failed menu read still reports the agents installed
  since the last one. A connect or test whose handshake could not read
  the menus keeps the ones already measured, and connecting an agent
  with a model refuses one its measured menu does not offer, as editing
  one already did.
- Retain the last model catalog when a refresh fails. Deleting a session
  prevents later writes, prompts, loads and route switches, waits for an
  already-running store write, and does not wait for unrelated catalog
  work. A turn, and its cancel, uses the catalog in hand instead of
  queueing behind another session's read.
- Relay questions and tool approvals through ACP. Content returned by
  another agent retains that agent's redaction policy, including values
  matching Raven's configured credentials. The rule covers Mirasim,
  shared sub-agent records and announcements, delegated tool results,
  A2A replies, logs, main-agent context, live ACP updates and session
  replay. Provenance is decided per result: a tool's declaration admits
  it to carrying another agent's content, and each result that does
  claims it, so a delegating tool's own errors, receipts and status
  lines stay Raven's text under the local scrub, and no result can claim
  provenance its tool never declared. Forwarding through `tool_call`
  carries a result whole for every tool, so a refusal reached that way
  ends the turn as a direct call does. The contract surface version is
  incremented for the added provenance fields. Local tool-result
  protections, outbound credential handling, permission checks and
  untrusted-content fencing retain their existing roles.
- Group unconfigured Mirasim presets into one card. Connect each member
  with its selected model in one write. Missing members open the
  existing installation and recheck flow, and a member whose handshake
  was refused for want of a sign-in is named in the drawer as it is on
  its card. A family whose members sit in two sections of the agents
  page draws one card in each. Narrow drawers keep the agent name and
  place the model and action on a second row.
- Guard command processes and clean up their descendants on
  cancellation, including ones a tool starts in a session of its own
  while the agent ignores SIGTERM: on Linux the guard adopts them as a
  child subreaper, and the process tree is read again before SIGKILL. A
  guard that outlasts its drain no longer replaces the command's own
  error with a shutdown traceback. Stage oversized prompts within the
  full Windows command-line budget, counting the venv launcher's second
  launch, refuse a `mirasim` that resolves to a `.bat` or `.cmd` shim,
  and preserve percent escapes and UNC authorities in file attachments.
  The guard starts without the ACP package, polls its command's exit
  quickly at first and then slowly, and the adapter's stderr log carries
  no local variables.
- Share the existing ACP stdin reader with the adapter, and update the
  integration docs, domain vocabulary, translations and agent-mark
  attribution.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Rebased onto current main and reran validation on Linux with Python
3.12.13 and Node 22.23.2. The isolated interpreter resolves this
checkout even when a subprocess clears PYTHONPATH. Existing dependencies
were reused without changing their declarations or lockfile. In the
commands below, `$CHECKS` denotes the temporary result directory used
for the run. Both Python selections use the same
`COVERAGE_FILE=$CHECKS/post-rebase.coverage`.

```sh
uv run --frozen --no-sync pytest tests/test_agent_loop_*.py tests/test_acp_mirasim_*.py tests/test_tool_registry_*.py tests/test_subagent_acp*.py tests/test_subagent_activity.py tests/test_subagent_history.py tests/test_subagent_manager.py tests/test_subagent_probe_state.py tests/test_security_untrusted_context.py tests/test_tool_admission.py tests/test_tool_search.py tests/test_cli_log_file.py tests/test_acp_journal.py tests/test_a2a_client_tool.py tests/test_subagent_mirasim.py tests/test_subagent_probe.py tests/test_subagent_third_party.py tests/test_ui_agent_marks.py tests/test_acp_replay.py tests/test_acp_updates.py tests/test_rpc_console.py tests/test_rpc_system.py -n 0 -x -q --tb=short --show-capture=no --cov=raven --cov-branch --cov-report=json:$CHECKS/post-rebase-coverage.json --cov-report=xml:$CHECKS/post-rebase-coverage.xml --junitxml=$CHECKS/post-rebase-python.xml
# 2871 passed
uv run --frozen --no-sync pytest tests/test_contracts_*.py tests/test_tool_admission.py tests/test_tool_registry_execute.py tests/test_security_untrusted_context.py tests/test_config_loader.py tests/test_raven_config_tool.py tests/test_rpc_subagents.py tests/test_subagent_name_vocabulary.py tests/test_tool_refusal_continuation.py -n 0 -q --tb=short --show-capture=no --cov=raven --cov-branch --cov-append --cov-report=json:$CHECKS/post-rebase-coverage.json --cov-report=xml:$CHECKS/post-rebase-coverage.xml
# 487 passed, 1 skipped (POSIX directory permissions are not enforced for root)
uv run --frozen --no-sync python scripts/coverage_gate.py --coverage-json $CHECKS/post-rebase-coverage.json diff --base-ref origin/main --head HEAD --threshold 90
# 93.33%, 1594/1708 executable changed lines; passed
```

The full Python suite, `uv run --frozen --python 3.12 --all-extras
pytest -q`: 9 failed, 32315 passed, 176 skipped, 2 xfailed. Each failing
id is then run by id against a clean archive of the merge base, through
the same interpreter and extras and with the imported `raven` checked to
resolve inside the archive; every id that fails here also fails there.

The catalog-only failure and resumed-session deletion regressions failed
before their fixes. Additional tests exercise deletion during catalog
reads, store writes and route lookups. The content-policy tests failed
against the former redaction behavior and now check exact content
through direct calls, tool_call, the main provider, recorded frames,
persisted transcripts and ACP replay. Local file reads and outgoing
credential records remain covered by controls.

In ui-web:

```sh
npm test -- --no-file-parallelism
# 250 files, 3977 tests passed
npm run gen:check
npm run type-check
npm run lint
npm run build
# passed; lint has four warnings in unchanged files
```

Chromium checks used the actual page and styles with synthetic agent
rows. Model selection travels in the connect request; Escape closes only
the picker; missing members reach the download link and Check again. At
390px and 320px the name columns measure 252px and 182px, respectively,
with no horizontal overflow in the drawer. No page errors were reported.

```sh
make check-commits check-large-files check-source-language
# passed
npm run lint:i18n --prefix ui-tui
npm run lint:rpc --prefix ui-tui
# passed
uv run --frozen --no-sync python -m pre_commit run --from-ref origin/main --to-ref HEAD
```

The full-PR pre-commit run passes except prettier (ui-tui), which flags
`src/components/newInstancePicker.tsx` and
`src/components/subagentsHub.tsx`. Both files are byte-identical to
main; running the formatter against their main contents fails too. The
fixes do not touch either file. Commit-scoped hooks for the new changes
pass. Ruff check and format, ty on changed production files, all ten
import contracts, and whitespace checks pass. Every replayed commit
retains its patch, and the PR title and body pass the repository's ASCII
and commit-message grammar checks.

The earlier native Windows verification covers prompt launch limits and
file-URI conversions. This follow-up has no new native Windows, real SMB
or live vendor/provider run.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Credential authorization and scope belong at the point where access is
granted. Redacting a later reply cannot revoke access the receiving
agent already has. Under this product policy, Raven preserves returned
content unchanged and leaves redaction decisions to the sending agent.
Credentials an agent includes in its reply can therefore remain in
Raven's retained logs and transcripts.

Agent-returned content is preserved according to the sender's policy
across all supported delegation paths. The adapter returns each reply
when the turn settles rather than streaming it. Short prompts use
command-line arguments; oversized prompts use temporary attachments.
Connecting an agent probes it: with a model chosen, on that model's own
route, so the pick a connect carries is proved before the row is stored;
with none, on the agent's default. Compatibility was originally measured
against Mirasim 0.0.465; older CLI versions have not been verified.

A stored Mirasim row whose interpreter no longer exists is rewritten in
the config file, once, by the first Raven to read the config after that
interpreter is gone. A row naming another interpreter that still exists
is left as written, so two installs reading one config do not rewrite it
back and forth.

Rollback is a revert. Configured Mirasim rows then need removal because
their bundled adapter is no longer present, and until they are removed
Raven refuses to load the config at all: an unknown preset name fails
validation, so the host does not start and the file has to be edited by
hand.

## Related Issues

N/A
2026-10-17 14:31:43 +08:00

233 lines
8.8 KiB
Python

"""Refusing a tool call and ending the turn are two decisions.
One flag used to answer both, and the agent loop did three things with it: it
refused the call, it cancelled the sibling calls the model had already written
in the same response, and it ended the turn without asking the model again. The
first two belong together -- a refused operation must not be reachable through a
sibling the model wrote before it knew the answer -- but the third is a separate
judgement, and no refusal could be expressed without it.
`blocks_call` and `continuation` are those two questions. Every refusal still
asks for `ABORT_TURN`, so nothing here asserts a behaviour change; what these
pin is that the loop reads the second field instead of inferring it from the
first.
"""
from __future__ import annotations
import pytest
from raven.agent.loop import AgentLoop
from raven.contracts.llm_provider import LLMResponse, ToolCallRequest
from raven.contracts.tool import Continuation, Tool, ToolOutput, ToolResult
from raven.spine.message import ChatType, Source
from raven.spine.turn import Origin, TurnRequest
def test_a_refusal_says_what_should_happen_to_the_turn() -> None:
"""The two questions, asked separately."""
refused = ToolResult(model_text="Error: blocked", blocks_call=True, continuation=Continuation.ABORT_TURN)
assert refused.blocks_call is True
assert refused.continuation is Continuation.ABORT_TURN
def test_an_ordinary_result_asks_for_neither() -> None:
ordinary = ToolResult(model_text="done")
assert ordinary.blocks_call is False
assert ordinary.continuation is Continuation.CONTINUE
def test_a_call_can_be_refused_without_ending_the_turn() -> None:
"""The shape the second cut needs. Nothing produces it yet -- what matters
here is that the type can express it, which one flag could not."""
refused = ToolResult(model_text="Error: blocked", blocks_call=True, continuation=Continuation.CONTINUE)
assert refused.blocks_call is True
assert refused.continuation is Continuation.CONTINUE
def test_the_registry_carries_both_across_its_boundary() -> None:
"""`ToolOutput` is what the loop actually reads, and it is a `str` subclass
-- so a field that is not copied across is silently absent rather than an
error."""
out = ToolOutput("Error: blocked", blocks_call=True, continuation=Continuation.ABORT_TURN)
assert out.blocks_call is True
assert out.continuation is Continuation.ABORT_TURN
@pytest.mark.parametrize("continuation", list(Continuation))
def test_every_continuation_survives_the_boundary(continuation: Continuation) -> None:
"""Parametrised over the enum rather than over two literals: a value added
later is covered by this test on the day it is added."""
out = ToolOutput("text", continuation=continuation)
assert out.continuation is continuation
# -- and the loop that reads them ---------------------------------------------
#
# The tests above pin the type. They cannot fail if the loop ignores the
# continuation entirely, which is exactly the mistake this cut is trying not to
# make, so the decision is exercised here through `_process_message`.
class _Refuser(Tool):
"""A tool that refuses, saying separately what should become of the turn.
No real tool asks for CONTINUE yet -- that is the second cut. A stub is how
the loop's half of the contract gets tested before a producer exists.
"""
def __init__(self, continuation: Continuation) -> None:
self._continuation = continuation
self.calls = 0
@property
def name(self) -> str:
return "refuser"
@property
def description(self) -> str:
return "refuses"
@property
def parameters(self) -> dict:
return {"type": "object", "properties": {}, "required": []}
async def execute(self, **kwargs) -> ToolResult:
self.calls += 1
return ToolResult(
model_text="Error: policy stopped this one.",
retryable=False,
blocks_call=True,
continuation=self._continuation,
)
class _Provider:
def __init__(self, *, parallel: bool = False) -> None:
calls = [ToolCallRequest(id="call-a", name="refuser", arguments={})]
if parallel:
calls.append(ToolCallRequest(id="call-b", name="refuser", arguments={}))
self.responses = [
LLMResponse(content="", tool_calls=calls, finish_reason="tool_calls"),
LLMResponse(content="Took another route.", finish_reason="stop"),
]
async def chat_with_retry(self, **kwargs) -> LLMResponse:
return self.responses.pop(0)
def get_default_model(self) -> str:
return "fake/model"
async def _run(agent: AgentLoop) -> str:
result, _media = await agent._process_message(
TurnRequest(
origin=Origin.USER,
source=Source(channel="tui", chat_id="default", sender_id="user", chat_type=ChatType.DM),
text="do the thing",
conversation="session-a",
),
session_key="session-a",
)
return result
async def test_the_loop_ends_the_turn_when_the_refusal_asks_for_it(tmp_path) -> None:
"""Today's only behaviour, unchanged by the rename."""
provider = _Provider()
agent = AgentLoop(provider=provider, workspace=tmp_path, model="fake/model")
agent.tools.register(_Refuser(Continuation.ABORT_TURN))
result = await _run(agent)
assert "no alternative method will be attempted" in result
# The second response is still on the shelf: the model was never asked again.
assert len(provider.responses) == 1
async def test_the_loop_asks_again_when_the_refusal_leaves_the_turn_alive(tmp_path) -> None:
"""The shape the second cut needs, read end to end.
The call is still refused and its result still reaches the model. What does
not happen is the runtime answering the user on the model's behalf.
"""
provider = _Provider()
agent = AgentLoop(provider=provider, workspace=tmp_path, model="fake/model")
agent.tools.register(_Refuser(Continuation.CONTINUE))
result = await _run(agent)
assert result == "Took another route."
assert provider.responses == []
async def test_a_blocked_call_cancels_its_siblings_even_when_the_turn_goes_on(tmp_path) -> None:
"""Blocking the call and ending the turn are separate, and the sibling
cancellation belongs to the first one: a call the model wrote before it knew
the answer must not run, whatever happens to the turn afterwards."""
provider = _Provider(parallel=True)
agent = AgentLoop(provider=provider, workspace=tmp_path, model="fake/model")
tool = _Refuser(Continuation.CONTINUE)
agent.tools.register(tool)
result = await _run(agent)
assert tool.calls == 1
assert result == "Took another route."
class _ForwardedProvider:
"""One refusal, reached through ``tool_call`` -- the route a tool the model
cannot see is invoked by. Its result travels back out through the forwarder,
which owns none of the fields the registry unwrapped."""
def __init__(self, forwarded: str) -> None:
self.responses = [
LLMResponse(
content="",
tool_calls=[
ToolCallRequest(id="call-a", name="tool_call", arguments={"name": forwarded, "arguments": {}})
],
finish_reason="tool_calls",
),
LLMResponse(content="Took another route.", finish_reason="stop"),
]
async def chat_with_retry(self, **kwargs) -> LLMResponse:
return self.responses.pop(0)
def get_default_model(self) -> str:
return "fake/model"
@pytest.mark.parametrize("withheld", [False, True])
async def test_a_forwarded_refusal_keeps_the_turn_decision_it_carried(tmp_path, withheld: bool) -> None:
"""Forwarding is a transport, not a filter.
``tool_call`` hands the target's result straight back, so the refusal's own
answer about the turn survives the extra hop exactly as it does on the
direct path. Before this, a forwarded refusal lost both flags and the loop
read it as ordinary output -- the model was asked again and the turn went on
after an operation the user had stopped.
"""
from raven.agent.tools.tool_search import ToolCallTool, ToolSearchController
forwarded = _Refuser(Continuation.ABORT_TURN)
provider = _ForwardedProvider("refuser")
agent = AgentLoop(provider=provider, workspace=tmp_path, model="fake/model")
agent.tools.register(forwarded)
controller = ToolSearchController(agent.tools, always_visible={"refuser"} if withheld else set())
agent.tools.register(ToolCallTool(controller))
result = await _run(agent)
assert forwarded.calls == 1
assert "no alternative method will be attempted" in result
# The second response is still on the shelf: the model was never asked again.
assert len(provider.responses) == 1