Raven-1/tests/test_utils_scratch.py
Dizhan Xue a4dfd6624e
fix(*): end a check's agent before removing its directory, so Windows connects pass (#974)
## Summary

On native Windows, connecting an agent that runs as a process chain
failed every time with `[WinError 32] The process cannot access the file
because it is being used by another process:
'...\raven_subagent_ping_xxxx'`. The Claude Code preset is one such
chain: `npx.cmd`, `cmd.exe`, the node adapter, then `claude.exe`.
`ping_agent` handed the agent a fresh temporary directory to launch in
and removed it when its `with` block ended, which was before the
`finally` that closes the ping's pool and so ends the agent's process
tree. Windows refuses to remove a directory that a live process has as
its working directory, even an empty one, and the broad `except
Exception` turned that removal error into the ping's verdict: an agent
that had answered was reported as not answering a test message. A ping
that timed out was misreported the same way, and each press left its
directory in the temp folder.

`verify_agent`, the handshake behind Test and behind the capability
record a Connect writes once its ping passes, removed its session
directory in the same order. The Claude Agent SDK starts `claude.exe` in
the session's directory, so that check came back `unknown` with `verify
failed: [WinError 32] ...`.

Ending the tree was already right: the Job Object from #951 ends every
process in it, including descendants started later. What was wrong was
when the directory was removed, and what a failed removal was taken to
mean.

- `raven/utils/scratch.py` adds `ScratchDir`, a temporary directory that
is removed only when its caller says the agent has ended. The removal is
tried up to 5 times, 0.2 s apart, while a tree that was just ended
finishes exiting. A directory that still cannot be removed is logged
with its path and left in the temp folder, never raised. It removes
through `TemporaryDirectory.cleanup`, so read-only entries are still
cleared on Windows as before.
- `ping_agent` removes its directory after `pool.close_all()`, and
`verify_agent` after `client.close()`. The CLI test (`run_test`) and the
Kimi Code print-mode ask use `ScratchDir` as an async context manager:
their process has ended inside the block, and a refused removal no
longer replaces the CLI's reply or raises out of the ask.
- The `Windows ACP stdio` CI job runs a new real-process test of both
checks against an agent of the reported shape.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Native Windows 11 (build 26300, Python 3.12) unless marked Linux.

Reproduction with a stand-in agent of the reported shape: a `.cmd`
wrapper starting a Python ACP agent that, on `session/new`, starts a
worker in the session's directory and waits for it before answering.

- `main`: the ping returns `ok=False` with `[WinError 32] ...
raven_subagent_ping_...`; the handshake returns `unknown` with `verify
failed: [WinError 32] ... raven_acp_verify_...`; both directories stay
in `%TEMP%`.
- This branch: the ping returns `ok=True` ("it ran and replied"), the
handshake `ready`, and nothing stays in `%TEMP%`. Over 20 rounds of both
checks, all 40 directories were removed on the first try after the
close.

```text
uv run --frozen --no-sync pytest tests/integration/test_subagent_connect_windows_real_process.py -m integration -q -n 0
```

1 passed. On `main` it fails with `[WinError 32] ...
raven_subagent_ping_...`, the error text in the issue.

New unit tests: 5 in `tests/test_utils_scratch.py`, 3 in
`tests/test_subagent_acp.py`, 4 in `tests/test_subagent_probe.py` (the
CLI one is POSIX-only, like the neighbours whose shell stand-in it
shares) and 1 in `tests/test_subagent_kimi_code.py`. The ping and
handshake ordering tests run the real pool, client and stub agent, with
Windows' refusal stood in on any host by `tests/_windows_dirs.py`: a
removal is refused while an agent the test launched is not yet closed.
With the three changed modules put back to `main`, the six call-site
tests fail, each with the symptom it names (the CLI one on Linux). Each
new construct was then removed on its own, and a test goes red for every
one: removal before the pool close, removal before the client close,
either `None` guard, raising after the last try, a single try, no pause
between tries, and `shutil.rmtree` in place of
`TemporaryDirectory.cleanup`. That last one only shows on Windows, where
a read-only file blocks the removal.

Slices of the suite that reach the changed modules, this branch against
`main` at a71b488e:

- Linux, in Docker (`python3.12-bookworm-slim`,
`--idle-ceiling-strict`), 20 test files plus the new one: 9 failed, 1718
passed, 88 errors on this branch; 8 failed, 1706 passed, 88 errors on
`main`. The errors and eight of the failures are the same ids on both,
in remote-machine and file-permission tests: the container has no `ps`
and runs as root. The ninth,
`test_a_ping_answers_from_the_machine_and_leaves_one_directory`, timed
out once starting the machine's daemon (`did not answer within 20s`)
while the host was under load; run alone it passed 3 of 3 times on both
sides.
- Windows, `tests/test_subagent_acp.py tests/test_subagent_probe.py
tests/test_subagent_kimi_code.py tests/test_subagent_host_env.py
tests/test_acp_windows.py tests/test_rpc_subagents.py
tests/test_subagent_github_copilot.py`: 57 failed, 635 passed, 4 skipped
on this branch; 61 failed, 619 passed, 3 skipped on `main`. No test
fails only on this branch. Four tests that fail on `main` under Windows
pass with this change,
`test_the_roster_and_the_connect_button_read_one_answer` among them.

For anyone reproducing on Windows: the suite isolates the home by
setting `HOME`, which `Path.home()` ignores there in favour of
`USERPROFILE`, so a run reads and writes the real `%USERPROFILE%\.raven`
and tests can see each other's records. The Windows numbers above were
taken with `USERPROFILE` pointed at an empty folder. That gap predates
this change and is not addressed here.

```text
uv run --frozen --no-sync ruff check <the 10 changed Python files>
uv run --frozen --no-sync ruff format --check <the 10 changed Python files>
uv run --frozen --no-sync ty check raven/utils/scratch.py raven/agent/subagent/probe.py raven/agent/subagent/kimi_code.py raven/acp_client/capabilities.py
uv run --frozen --no-sync lint-imports
```

All passed, 10 import contracts kept.

Not measured: the real `@agentclientprotocol/claude-agent-acp` adapter.
The stand-in above takes its place.

- [ ] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

The tests box stays unchecked because the slices above carry failures
that `main` carries too. No user-facing text or docs change.

## Risk

- POSIX behaviour is unchanged apart from order: the directory is
removed on the first try, as before, now after the agent has ended
rather than before.
- On Windows a refused removal adds at most about 0.8 s to a check, and
the directory is then left in the temp folder with a warning that names
it. Before this change every such check left its directory there and
failed.
- A check whose agent leaves a process running outside its job, or a CLI
that leaves a background worker, now reports the agent's answer where it
used to report the removal error.
- Rollback: revert the commit. No configuration, schema or stored state
changes.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

Fixes #969

Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-19 10:35:48 +08:00

100 lines
3.3 KiB
Python

"""A temporary directory an agent worked in goes once the agent has, and never takes a verdict with it."""
from __future__ import annotations
import stat
import time
from pathlib import Path
import pytest
from loguru import logger
from raven.utils import scratch
from raven.utils.scratch import ScratchDir
from tests._windows_dirs import scratch_root, windows_removal
@pytest.fixture(autouse=True)
def _root(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
return scratch_root(tmp_path, monkeypatch)
async def test_the_directory_and_what_was_left_in_it_go_when_the_block_ends(_root: Path) -> None:
async with ScratchDir("raven_check_") as path:
assert path.parent == _root and path.name.startswith("raven_check_")
(path / "nested").mkdir()
(path / "nested" / "left.txt").write_text("x", encoding="utf-8")
assert not path.exists()
assert list(_root.iterdir()) == []
async def test_a_read_only_file_left_in_it_does_not_keep_the_directory() -> None:
"""Windows will not delete a read-only file, and git marks its objects so.
An agent that ran ``git init`` where it was put leaves some. POSIX deletes
them regardless, so this case only bites on Windows.
"""
async with ScratchDir("raven_check_") as path:
kept = path / "object"
kept.write_text("x", encoding="utf-8")
kept.chmod(stat.S_IREAD)
assert not path.exists()
async def test_a_directory_still_in_use_is_tried_again_until_the_process_lets_go(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Ending a process tree returns before the last of it has exited, so a first try can be refused."""
made = ScratchDir("raven_check_")
asked: list[Path] = []
def holding(path: Path) -> bool:
asked.append(path)
return len(asked) < 3
refused = windows_removal(monkeypatch, holding)
await made.remove()
assert refused == [made.path, made.path]
assert not made.path.exists()
async def test_the_tries_are_spread_over_the_time_a_tree_takes_to_exit(monkeypatch: pytest.MonkeyPatch) -> None:
"""Tries made back to back all land inside the exit they are there to wait out."""
monkeypatch.setattr(scratch, "_REMOVE_PAUSE_S", 0.05)
made = ScratchDir("raven_check_")
first_try: list[float] = []
def holding(path: Path) -> bool:
first_try.append(first_try[0] if first_try else time.monotonic())
return time.monotonic() < first_try[0] + 0.08
refused = windows_removal(monkeypatch, holding)
await made.remove()
assert refused, "the first try was not refused, so this tested nothing"
assert not made.path.exists()
async def test_a_directory_that_stays_in_use_is_left_and_logged_rather_than_raised(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The caller reports an agent's answer, and a removal failure raised in its place says the agent failed."""
made = ScratchDir("raven_check_")
refused = windows_removal(monkeypatch, lambda path: path == made.path)
warnings: list[str] = []
sink = logger.add(lambda message: warnings.append(message.record["message"]), level="WARNING")
try:
await made.remove()
finally:
logger.remove(sink)
assert len(refused) == scratch._REMOVE_ATTEMPTS
assert made.path.is_dir()
assert len(warnings) == 1
assert str(made.path) in warnings[0]