Raven-1/tests/test_cli_log_file.py
Dizhan Xue 547c1aba5e
feat(*): reach Mirasim's agents as ACP sub-agent presets (#971)
## Summary

Connect the coding agents Mirasim runs as Raven ACP sub-agents, with one
bundled preset per agent and no separate adapter download.

- Serve Claude Code, Codex, Kimi Code, Antigravity, Qwen Code, Grok
  Build, DSH, Pi and ZCode CLI through `mirasim ui-cli`. The adapter
  runs on Raven's interpreter, so a Mirasim preset is not offered on a
  registered machine, and a stored row whose interpreter is gone is
  pointed at the running one when the config is next read. It withholds
  session MCP because ui-cli does not accept per-session servers. The
  boot capability backfill asks Mirasim once which agents it has
  installed and handshakes only those.
- Read model and effort choices from Mirasim. Offer the own models of
  Claude Code, Codex, Kimi Code and ZCode CLI on both their own-account
  and Mirasim platform routes. Persist conversation bindings and switch
  routes before continued turns. A menu read that fails is tried again
  after a short pause rather than kept for the cached copy's lifetime,
  an effort the menus in hand do not offer is read again before it is
  refused, and a failed menu read still reports the agents installed
  since the last one. A connect or test whose handshake could not read
  the menus keeps the ones already measured, and connecting an agent
  with a model refuses one its measured menu does not offer, as editing
  one already did.
- Retain the last model catalog when a refresh fails. Deleting a session
  prevents later writes, prompts, loads and route switches, waits for an
  already-running store write, and does not wait for unrelated catalog
  work. A turn, and its cancel, uses the catalog in hand instead of
  queueing behind another session's read.
- Relay questions and tool approvals through ACP. Content returned by
  another agent retains that agent's redaction policy, including values
  matching Raven's configured credentials. The rule covers Mirasim,
  shared sub-agent records and announcements, delegated tool results,
  A2A replies, logs, main-agent context, live ACP updates and session
  replay. Provenance is decided per result: a tool's declaration admits
  it to carrying another agent's content, and each result that does
  claims it, so a delegating tool's own errors, receipts and status
  lines stay Raven's text under the local scrub, and no result can claim
  provenance its tool never declared. Forwarding through `tool_call`
  carries a result whole for every tool, so a refusal reached that way
  ends the turn as a direct call does. The contract surface version is
  incremented for the added provenance fields. Local tool-result
  protections, outbound credential handling, permission checks and
  untrusted-content fencing retain their existing roles.
- Group unconfigured Mirasim presets into one card. Connect each member
  with its selected model in one write. Missing members open the
  existing installation and recheck flow, and a member whose handshake
  was refused for want of a sign-in is named in the drawer as it is on
  its card. A family whose members sit in two sections of the agents
  page draws one card in each. Narrow drawers keep the agent name and
  place the model and action on a second row.
- Guard command processes and clean up their descendants on
  cancellation, including ones a tool starts in a session of its own
  while the agent ignores SIGTERM: on Linux the guard adopts them as a
  child subreaper, and the process tree is read again before SIGKILL. A
  guard that outlasts its drain no longer replaces the command's own
  error with a shutdown traceback. Stage oversized prompts within the
  full Windows command-line budget, counting the venv launcher's second
  launch, refuse a `mirasim` that resolves to a `.bat` or `.cmd` shim,
  and preserve percent escapes and UNC authorities in file attachments.
  The guard starts without the ACP package, polls its command's exit
  quickly at first and then slowly, and the adapter's stderr log carries
  no local variables.
- Share the existing ACP stdin reader with the adapter, and update the
  integration docs, domain vocabulary, translations and agent-mark
  attribution.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Rebased onto current main and reran validation on Linux with Python
3.12.13 and Node 22.23.2. The isolated interpreter resolves this
checkout even when a subprocess clears PYTHONPATH. Existing dependencies
were reused without changing their declarations or lockfile. In the
commands below, `$CHECKS` denotes the temporary result directory used
for the run. Both Python selections use the same
`COVERAGE_FILE=$CHECKS/post-rebase.coverage`.

```sh
uv run --frozen --no-sync pytest tests/test_agent_loop_*.py tests/test_acp_mirasim_*.py tests/test_tool_registry_*.py tests/test_subagent_acp*.py tests/test_subagent_activity.py tests/test_subagent_history.py tests/test_subagent_manager.py tests/test_subagent_probe_state.py tests/test_security_untrusted_context.py tests/test_tool_admission.py tests/test_tool_search.py tests/test_cli_log_file.py tests/test_acp_journal.py tests/test_a2a_client_tool.py tests/test_subagent_mirasim.py tests/test_subagent_probe.py tests/test_subagent_third_party.py tests/test_ui_agent_marks.py tests/test_acp_replay.py tests/test_acp_updates.py tests/test_rpc_console.py tests/test_rpc_system.py -n 0 -x -q --tb=short --show-capture=no --cov=raven --cov-branch --cov-report=json:$CHECKS/post-rebase-coverage.json --cov-report=xml:$CHECKS/post-rebase-coverage.xml --junitxml=$CHECKS/post-rebase-python.xml
# 2871 passed
uv run --frozen --no-sync pytest tests/test_contracts_*.py tests/test_tool_admission.py tests/test_tool_registry_execute.py tests/test_security_untrusted_context.py tests/test_config_loader.py tests/test_raven_config_tool.py tests/test_rpc_subagents.py tests/test_subagent_name_vocabulary.py tests/test_tool_refusal_continuation.py -n 0 -q --tb=short --show-capture=no --cov=raven --cov-branch --cov-append --cov-report=json:$CHECKS/post-rebase-coverage.json --cov-report=xml:$CHECKS/post-rebase-coverage.xml
# 487 passed, 1 skipped (POSIX directory permissions are not enforced for root)
uv run --frozen --no-sync python scripts/coverage_gate.py --coverage-json $CHECKS/post-rebase-coverage.json diff --base-ref origin/main --head HEAD --threshold 90
# 93.33%, 1594/1708 executable changed lines; passed
```

The full Python suite, `uv run --frozen --python 3.12 --all-extras
pytest -q`: 9 failed, 32315 passed, 176 skipped, 2 xfailed. Each failing
id is then run by id against a clean archive of the merge base, through
the same interpreter and extras and with the imported `raven` checked to
resolve inside the archive; every id that fails here also fails there.

The catalog-only failure and resumed-session deletion regressions failed
before their fixes. Additional tests exercise deletion during catalog
reads, store writes and route lookups. The content-policy tests failed
against the former redaction behavior and now check exact content
through direct calls, tool_call, the main provider, recorded frames,
persisted transcripts and ACP replay. Local file reads and outgoing
credential records remain covered by controls.

In ui-web:

```sh
npm test -- --no-file-parallelism
# 250 files, 3977 tests passed
npm run gen:check
npm run type-check
npm run lint
npm run build
# passed; lint has four warnings in unchanged files
```

Chromium checks used the actual page and styles with synthetic agent
rows. Model selection travels in the connect request; Escape closes only
the picker; missing members reach the download link and Check again. At
390px and 320px the name columns measure 252px and 182px, respectively,
with no horizontal overflow in the drawer. No page errors were reported.

```sh
make check-commits check-large-files check-source-language
# passed
npm run lint:i18n --prefix ui-tui
npm run lint:rpc --prefix ui-tui
# passed
uv run --frozen --no-sync python -m pre_commit run --from-ref origin/main --to-ref HEAD
```

The full-PR pre-commit run passes except prettier (ui-tui), which flags
`src/components/newInstancePicker.tsx` and
`src/components/subagentsHub.tsx`. Both files are byte-identical to
main; running the formatter against their main contents fails too. The
fixes do not touch either file. Commit-scoped hooks for the new changes
pass. Ruff check and format, ty on changed production files, all ten
import contracts, and whitespace checks pass. Every replayed commit
retains its patch, and the PR title and body pass the repository's ASCII
and commit-message grammar checks.

The earlier native Windows verification covers prompt launch limits and
file-URI conversions. This follow-up has no new native Windows, real SMB
or live vendor/provider run.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Credential authorization and scope belong at the point where access is
granted. Redacting a later reply cannot revoke access the receiving
agent already has. Under this product policy, Raven preserves returned
content unchanged and leaves redaction decisions to the sending agent.
Credentials an agent includes in its reply can therefore remain in
Raven's retained logs and transcripts.

Agent-returned content is preserved according to the sender's policy
across all supported delegation paths. The adapter returns each reply
when the turn settles rather than streaming it. Short prompts use
command-line arguments; oversized prompts use temporary attachments.
Connecting an agent probes it: with a model chosen, on that model's own
route, so the pick a connect carries is proved before the row is stored;
with none, on the agent's default. Compatibility was originally measured
against Mirasim 0.0.465; older CLI versions have not been verified.

A stored Mirasim row whose interpreter no longer exists is rewritten in
the config file, once, by the first Raven to read the config after that
interpreter is gone. A row naming another interpreter that still exists
is left as written, so two installs reading one config do not rewrite it
back and forth.

Rollback is a revert. Configured Mirasim rows then need removal because
their bundled adapter is no longer present, and until they are removed
Raven refuses to load the config at all: an unknown preset name fails
validation, so the host does not start and the file has to be edited by
hand.

## Related Issues

N/A
2026-10-17 14:31:43 +08:00

318 lines
12 KiB
Python

"""Unit tests for the shared loguru→file redirect helper.
Covers parameterization (filename / file_level / terminal_level / RAVEN_CLI_DEBUG)
and the stdlib-logging interception. The third-party TTY-handler stripping
has its own real-litellm regression guard in
``test_cli_tui_logging_isolation.py``.
"""
from __future__ import annotations
import logging
import logging.handlers
from pathlib import Path
import pytest
from raven.cli._log_file import redirect_loguru_to_file
from raven.config.loader import set_config_path
@pytest.fixture
def tmp_logs(tmp_path: Path, monkeypatch):
"""Point get_logs_dir() at a tmp instance + restore global log state.
redirect_loguru_to_file() mutates process-global logging: it installs a
root InterceptHandler and strips stderr StreamHandlers off every named
logger. Snapshot root + all named-logger handlers so this fixture does not
leak that mutation into sibling tests (notably the litellm-based
regression guard).
"""
set_config_path(tmp_path / "config.json")
monkeypatch.delenv("RAVEN_CLI_DEBUG", raising=False)
root = logging.getLogger()
saved_handlers = list(root.handlers)
saved_filters = list(root.filters)
saved_level = root.level
manager = logging.root.manager
saved_named = {name: list(lg.handlers) for name, lg in manager.loggerDict.items() if isinstance(lg, logging.Logger)}
try:
yield tmp_path / "logs"
finally:
from loguru import logger
logger.remove()
logger.configure(patcher=None)
root.handlers = saved_handlers
root.filters = saved_filters
root.level = saved_level
for name, handlers in saved_named.items():
lg = manager.loggerDict.get(name)
if isinstance(lg, logging.Logger):
lg.handlers = handlers
set_config_path(None) # type: ignore[arg-type]
def _flush_and_read(path: Path) -> str:
"""Remove sinks (joins the enqueued file thread → flush) then read."""
from loguru import logger
logger.remove()
return path.read_text() if path.exists() else ""
def test_writes_to_named_file_under_logs_dir(tmp_logs: Path) -> None:
from loguru import logger
log_path = redirect_loguru_to_file("gateway.log")
assert log_path == tmp_logs / "gateway.log"
logger.info("hello-gateway-marker")
assert "hello-gateway-marker" in _flush_and_read(log_path)
def test_agent_content_keeps_its_query_values_in_the_log(tmp_logs: Path) -> None:
from loguru import logger
log_path = redirect_loguru_to_file("gateway.log")
external = "Agent returned https://agent.invalid/result?token=agent-example"
logger.bind(agent_output=True).info(external)
logger.info("Host requested https://host.invalid/api?token=host-example")
content = _flush_and_read(log_path)
assert external in content
assert "host-example" not in content
assert "https://host.invalid/api?token=<redacted>" in content
def test_terminal_warning_keeps_warnings_drops_info(tmp_logs: Path, capsys) -> None:
from loguru import logger
redirect_loguru_to_file("gateway.log", terminal_level="WARNING")
logger.info("info-not-on-terminal")
logger.warning("warn-on-terminal")
err = capsys.readouterr().err
assert "warn-on-terminal" in err
assert "info-not-on-terminal" not in err
def test_terminal_none_silences_stderr(tmp_logs: Path, capsys) -> None:
from loguru import logger
redirect_loguru_to_file("tui.log", terminal_level=None)
logger.warning("should-not-appear-on-terminal")
assert "should-not-appear-on-terminal" not in capsys.readouterr().err
def test_record_filter_drops_matching_records(tmp_logs: Path) -> None:
"""record_filter lets a caller drop sink-specific noise (the TUI
suppresses watchfiles spam) without it reaching the file."""
from loguru import logger
def _drop_spam(record: dict) -> bool:
return "rust notify timeout" not in record["message"]
log_path = redirect_loguru_to_file("tui.log", record_filter=_drop_spam)
logger.info("rust notify timeout, continuing")
logger.info("real-turn-activity-marker")
text = _flush_and_read(log_path)
assert "real-turn-activity-marker" in text
assert "rust notify timeout" not in text
def test_record_filter_none_keeps_all(tmp_logs: Path) -> None:
"""Default (gateway) path: no filter → every record kept."""
from loguru import logger
log_path = redirect_loguru_to_file("gateway.log")
logger.info("rust notify timeout, continuing")
assert "rust notify timeout" in _flush_and_read(log_path)
def test_env_debug_mirrors_to_stderr(tmp_logs: Path, capsys, monkeypatch) -> None:
from loguru import logger
monkeypatch.setenv("RAVEN_CLI_DEBUG", "1")
redirect_loguru_to_file("gateway.log", terminal_level=None)
logger.debug("debug-mirror-marker")
assert "debug-mirror-marker" in capsys.readouterr().err
def test_file_level_filters_below_threshold(tmp_logs: Path) -> None:
from loguru import logger
log_path = redirect_loguru_to_file("gateway.log", file_level="WARNING")
logger.info("info-below-threshold")
logger.warning("warning-at-threshold")
content = _flush_and_read(log_path)
assert "warning-at-threshold" in content
assert "info-below-threshold" not in content
def test_stdlib_logging_intercepted_to_file(tmp_logs: Path) -> None:
log_path = redirect_loguru_to_file("gateway.log", file_level="DEBUG")
logging.getLogger("some.third.party").warning("stdlib-intercepted-marker")
assert "stdlib-intercepted-marker" in _flush_and_read(log_path)
def test_file_sink_does_not_dump_local_variable_values(tmp_logs: Path) -> None:
"""diagnose=False: a secret bound in the failing frame must not be written
into the persisted file via loguru's variable-annotated traceback."""
from loguru import logger
log_path = redirect_loguru_to_file("gateway.log", file_level="DEBUG")
api_token = "SECRET-TOKEN-9z9z9z"
try:
# api_token is referenced on the raising line, so diagnose=True WOULD
# annotate it with its value; the exception message itself is "boom".
raise RuntimeError(api_token[:0] or "boom")
except RuntimeError:
logger.exception("processing failed")
content = _flush_and_read(log_path)
assert "processing failed" in content
assert "SECRET-TOKEN-9z9z9z" not in content
def test_terminal_sink_does_not_dump_local_variable_values(tmp_logs: Path, capsys) -> None:
"""The mirror of the test above, for the sink two lines below it in the source.
The file sink was configured ``backtrace=False, diagnose=False`` and said why;
the stderr sink beside it took loguru's defaults, which are both ``True``.
Measured before the fix: a token bound in the failing frame appeared six times
across twenty-seven lines of stderr.
stderr matters more here rather than less. ``raven acp`` runs as an editor's
subprocess and the editor displays that stream, so a traceback annotated with
every local's value is rendered into somebody's UI rather than into a file
only they can read.
"""
from loguru import logger
redirect_loguru_to_file("gateway.log", terminal_level="ERROR")
api_token = "SECRET-TOKEN-8y8y8y"
try:
raise RuntimeError(api_token[:0] or "boom")
except RuntimeError:
logger.exception("processing failed")
err = capsys.readouterr().err
assert "processing failed" in err, "the record itself must still reach the terminal"
assert "SECRET-TOKEN-8y8y8y" not in err
# A frame-by-frame annotated traceback is also just noise on a protocol
# channel's neighbour; the interpreter's own excepthook still prints the
# plain one, so nothing diagnosable is lost.
assert err.count("\n") < 20, f"the terminal traceback should be compact, got:\n{err}"
# ---------------------------------------------------------------------------
# Root-logger TTY StreamHandler stripping
# ---------------------------------------------------------------------------
def test_strip_tty_stream_handlers_removes_root_stdout_handler(tmp_logs: Path) -> None:
"""_strip_tty_stream_handlers() must remove a StreamHandler(sys.stdout)
installed on the ROOT logger (not just on named loggers)."""
import sys
from raven.cli._log_file import _strip_tty_stream_handlers
root = logging.getLogger()
stdout_handler = logging.StreamHandler(sys.stdout)
root.addHandler(stdout_handler)
try:
_strip_tty_stream_handlers()
assert stdout_handler not in root.handlers, "_strip_tty_stream_handlers must remove root stdout StreamHandler"
finally:
root.removeHandler(stdout_handler)
def test_strip_tty_stream_handlers_keeps_root_non_tty_handler(tmp_logs: Path) -> None:
"""_strip_tty_stream_handlers() must NOT remove a non-TTY handler (e.g. a
MemoryHandler) from the ROOT logger."""
from raven.cli._log_file import _strip_tty_stream_handlers
root = logging.getLogger()
mem_handler = logging.handlers.MemoryHandler(capacity=10)
root.addHandler(mem_handler)
try:
_strip_tty_stream_handlers()
assert mem_handler in root.handlers, "_strip_tty_stream_handlers must not remove non-TTY root handlers"
finally:
root.removeHandler(mem_handler)
# ---------------------------------------------------------------------------
# Credentials must not reach the persisted file
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
"query",
[
"?engine=google&q=cats&api_key=SECRET-VALUE-1",
"?api_key=SECRET-VALUE-1&engine=google",
"?api_key=SECRET-VALUE-1",
"?apiKey=SECRET-VALUE-1&q=cats",
"?q=cats&token=SECRET-VALUE-1",
],
)
def test_a_url_borne_credential_never_reaches_the_file(tmp_logs: Path, query: str) -> None:
"""Driven through real httpx rather than a hand-written line: the record
that leaks is httpx's own success-path INFO, so its wording is what the
pattern has to match, and only httpx can say what that wording is.
The level is set here rather than inherited: importing litellm anywhere in
the process raises the ``httpx`` logger to WARNING as an import side
effect, which would leave every assertion below vacuously true.
"""
import asyncio
import httpx
httpx_logger = logging.getLogger("httpx")
saved_level = httpx_logger.level
log_path = redirect_loguru_to_file("gateway.log", file_level="INFO", terminal_level=None)
async def _get() -> None:
transport = httpx.MockTransport(lambda request: httpx.Response(200, json={}))
async with httpx.AsyncClient(transport=transport) as client:
await client.get(f"https://vendor.test/search{query}")
try:
httpx_logger.setLevel(logging.INFO)
asyncio.run(_get())
text = _flush_and_read(log_path)
finally:
httpx_logger.setLevel(saved_level)
assert "HTTP Request" in text, "the httpx record itself must still be persisted"
assert "SECRET-VALUE-1" not in text
assert "<redacted>" in text
def test_redaction_keeps_the_rest_of_the_url(tmp_logs: Path) -> None:
from loguru import logger
log_path = redirect_loguru_to_file("gateway.log", file_level="INFO", terminal_level=None)
logger.error("WebFetch error for {}: {}", "https://vendor.test/v1?q=cats&api_key=SECRET-VALUE-2", "boom")
text = _flush_and_read(log_path)
assert "SECRET-VALUE-2" not in text
assert "https://vendor.test/v1?q=cats&api_key=<redacted>" in text
assert "boom" in text
def test_a_line_carrying_no_credential_is_untouched(tmp_logs: Path) -> None:
from loguru import logger
log_path = redirect_loguru_to_file("gateway.log", file_level="INFO", terminal_level=None)
logger.info("channel=whatsapp status=ready keys=3")
assert "channel=whatsapp status=ready keys=3" in _flush_and_read(log_path)