## Summary Raven-Code reports Python syntax errors after file edits but misses unresolved names and equivalent feedback after shell edits. Add advisory Pyflakes F821/F822/F823 checks to the existing tool result; optional F401 notices remain off by default. No additional model call or code execution is involved. ```text write_file / edit_file -> inspect final source, compare with a baseline that can be read and analyzed -> retain syntax warnings; suppress existing name findings exec -> require checkpoint visibility for the working directory -> read the changed files from the host's write record for the command -> ask visibility in batches of 256, select at most 8, check up to 4 concurrently -> append ordered diagnostics, preserving exit status and file metadata ``` Existing syntax failures remain visible after unrelated edits and identical rewrites. A baseline that cannot be read, parsed or analyzed is marked unknown; overwriting non-UTF-8 source still checks the known replacement. Source nested too deeply for the parser or checker, and a changed file that cannot be read, report diagnostics unavailable for that file rather than passing as clean. Every check reads a file as the bytes Python runs: text a write or edit stores is checked as its UTF-8 bytes, the content it replaced included, and a file from disk as its bytes, each decoded as Python decodes source, by its coding declaration or as UTF-8. Bytes that do not decode are reported as a syntax error at the line where decoding stops, so a declaration the written text cannot satisfy is reported however the file was changed. Noqa comments are read as Ruff reads them, including after other comment directives and never inside string literals: a file-level `ruff: noqa` / `flake8: noqa` counts only on a line of its own, and a marker counts only when codes follow its colon or, with no colon, whitespace, another comment or the end of the comment follows it. Unlike Ruff, which takes the first directive in a comment, every directive in it counts. The seeded tool guide gives the syntax note's own advice. With no checkpoint or no visibility verdict, exec diagnostics do not inspect changed file contents; direct file checks remain available. Pyflakes is imported when the tool face loads, not by whichever check first needs it: imported lazily, its cold import fell inside that check's one-second wait, and on a slow Windows runner the one test that paid it reported diagnostics unavailable for a plain syntax error. Private workers bound reads and checks to four active jobs per process. Saturated capacity rejects new work without queuing, and timeout/cancellation retains each slot until the actual work finishes. Name checks are capped at 256 KiB of source and the syntax check at 2 MiB, and a check skipped for size says so in one line. Findings are capped at eight per file, and exec diagnostic text at 6,000 characters plus omission notices. Runtime changes stay inside the Raven-Code code-flow plugin, except for one fix to `CheckpointService.trackable`, the visibility verdict that exec diagnostics and the host's exec write record both rely on. On macOS, git precomposes a file name it reads off the disk before matching an ignore rule, but takes the paths `trackable` sends to `git check-ignore --stdin` as written. A file stored with a decomposed (NFD) name under a `.gitignore` rule written precomposed (NFC) is therefore left out of the checkpoint, yet main vouches for it, so the write record's diff carries its text, and exec diagnostics would quote its source lines to the model. Nor is git's precomposition NFC: it composes only neighbouring characters, by Unicode 3.2's table, and reorders nothing, so a name whose combining marks are out of canonical order comes out as neither the name as given nor its NFC. `trackable` now asks about each name as given, as NFC and in git's macOS spelling, and withholds it when any spelling is ignored. Other shared file/shell/snapshot implementations are unchanged. Raven-Code has no separate dependency manifest, so Pyflakes uses the existing root dependencies. It is the only added package; uv normalized existing lockfile markers without changing other package versions. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - **Before:** on main, Raven-Code's file tools report Python syntax errors only, and its exec reports nothing about the Python files a command changed. On macOS, main's `CheckpointService.trackable` vouches for a file stored as `secret_cafe` + U+0301 + `.py` under the rule `secret_caf` + U+00E9 + `.py`: the user's own git ignores it and the checkpoint's staged tree leaves it out, yet its text reaches the exec write record's stored diff. - **After:** on macOS 26.6.2 / arm64 with Apple Git 2.54.0, the command below passes 364 tests on Python 3.12.13 and on 3.14.6. The full default suite on 3.12.13 has 32283 passed, 169 skipped, 20 failed and 3 errors over 22 test ids: 21 of them also fail on the exact base in the same environment, and the other, a remote process-tree cancel test that missed its deadline under the suite's load, passes when run alone. Tests cover actual worker overlap, retained capacity after timeout/cancellation, host-tool isolation, hidden-file quotas, visibility batching through the host's write-record wiring, baseline handling, parser-limit failures, the size bounds and their skip notes, undecodable bytes placed where decoding stops, coding declarations in written and edited text, the baseline read staying on a worker, the checker loaded with its module, the file each unavailable note names, comment directives and their marker boundary, the tool guide's agreement with the syntax note, portable commands in paths with spaces/non-ASCII characters, and the checkpoint's verdict on decomposed names and on names git spells as neither given nor NFC, held against its own staged tree. Removing any of the three spellings from the query fails the cases that pin it, as does removing the marker boundary, the parser-limit report, the unknown-baseline handling, the syntax check past the name-check bound and the decoding report, checking written text as its stored bytes, the worker around the baseline read, loading the checker with its module, or the file name in an unavailable note; restoring the guide's earlier advice fails its agreement test. git's macOS spelling is what libiconv's UTF-8-MAC conversion returns, which git precomposes with: composing neighbours by Unicode 3.2's table reproduced it for all 281,928 names built from every decomposable character's decomposition (reordered, partly composed, or with a mark added), and across 784,220 further mixed names and pairs no name was spelled outside the three asked; the same held under Python 3.13 and 3.14. The decomposed-name check asserts the macOS outcome only where git precomposes names, so it also passes with `core.precomposeunicode = false` set through `GIT_CONFIG_GLOBAL`. Across 102 noqa comment variants the reported lines match Ruff 0.15.12, except the three that hold several directives in one comment. Through the real `AgentLoop`, code-flow tool face, `CodeExecutor` and checkpoint, with the working directory under `/tmp`, under `$TMPDIR` and in a path with spaces and CJK characters, exec diagnostics report created, modified and decomposed-name files, follow `working_dir`, report an undecodable file at the line where decoding stops, a syntax error past 256 KiB and a skip past 2 MiB, and withhold `.gitignore`d files, including decomposed names and names git spells as neither given nor NFC; across repeated runs on both interpreters, a file near the 256 KiB cap is checked in 52-135 ms (117-128 ms against a baseline), the syntax of a file near 2 MiB in 318-371 ms, and 600 ignored files plus one visible file take 73-90 ms with the event loop held at most 20 ms. At this head CI passes the unit suite on Linux/Python 3.12 (32619 tests over four shards), and its two Windows steps pass the diagnostic files on native Windows/Python 3.13: 122 file/worker tests, and 27 exec diagnostic tests with 17 pre-existing executor tests deselected; the symlink fixture skips only where the host cannot create links. ```sh env -u SERPER_API_KEY -u RAVEN_MIRRORED_SERPER_API_KEY uv run --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_flow_exec.py tests/test_agents_code_flow_plugin.py tests/test_agents_code_flow_read_state.py tests/test_agents_code_launcher.py tests/test_workdir_snapshot.py tests/test_shell_command_writes.py tests/test_runtime_checkpoint.py tests/test_trajectory_upload.py::test_blocked_upload_hits_the_deadline -o addopts= -q --tb=short --maxfail=3 uv run --no-sync ruff check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_diagnostic_workers.py uv run --no-sync ty check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py uv run --no-sync lint-imports uv run --no-sync ruff format --check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py ``` Native Windows CI commands: ```sh uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py -q -n 0 --tb=short uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_flow_exec.py -k test_exec_ -q -n 0 --tb=short ``` Ruff, format and type checks passed; all 10 import contracts kept; the commit-message, large-file and source-language checks pass over the branch. The launcher tests unset inherited search API keys to keep the expected tool list deterministic. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed ## Risk Scope: Raven-Code's enabled code-flow tool face; installing Pyflakes does not activate checks in other agents. The change is reversible and never fixes/removes code or rolls back edits. Static diagnostics can need judgment for dynamic names, target Python versions, and project lint configuration; only explicit noqa directives are honored. Passing diagnostics do not replace tests. The `trackable` fix reaches every exec write record, not only Raven-Code: a file whose name, as given, as NFC or in git's macOS spelling, matches an ignore rule keeps its counts and loses its diff and removal body. Where git matches another of those spellings, as off macOS for a decomposed name under a precomposed rule, the checkpoint stores the file but it is still withheld; its counts remain. Exec diagnostics now require an active checkpoint. Noninteractive runs using the default interactive checkpoint policy must explicitly enable checkpoints to receive this feedback. Visibility is checked in batches of up to 256 files and stops once eight visible files are found; each batch holds the event loop while the host resolves its paths, about 130 ms for a full batch on native Windows. Each file-check wait is one second, and post-command diagnostics have a three-second wait budget; past it, or when they fail as a whole, they are dropped and the command's result is kept. These are caller budgets, not hard I/O deadlines: running OS reads and inherited synchronous host callbacks cannot be forcibly interrupted. Private worker capacity remains bounded. Between 256 KiB and 2 MiB only syntax is checked: near 2 MiB that took 0.3-0.4 s and about 190 MiB here, up to four checks run at once, and on slower hardware such a check can outlast the one-second wait and report diagnostics unavailable. Python's GIL means concurrency does not promise a CPU speedup. Windows CI exercises the diagnostics layer with the real native DirectExecutor. The existing CodeExecutor environment probe uses POSIX quoting and already fails on Windows at the base; this PR neither changes that probe nor claims end-to-end compatibility for it. Symlink setup is skipped only if the host cannot create the fixture. Disable `pythonLint` for syntax-only feedback, `execPythonDiagnostics` for no post-command checks, or `pythonSyntaxNote` for no diagnostics, under `plugins.config["code-flow"].tools`. The PR can also be reverted as a unit. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues N/A --------- Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn> Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
1270 lines
48 KiB
Python
1270 lines
48 KiB
Python
"""Per-turn shadow-git checkpoint + max-iter interrupted handling.
|
||
|
||
Covers the runtime-discipline safety net gated by
|
||
``config.runtime.checkpoint.policy`` × ``AgentLoop(interactive=...)``:
|
||
- CheckpointService snapshots the worktree without touching the user's .git.
|
||
- A max-iteration turn reports ``status="interrupted"`` (not a fake
|
||
completion) and the workspace is snapshotted for recovery.
|
||
- With policy="never" (or "interactive" + interactive=False), the loop is
|
||
behaviorally unchanged from baseline.
|
||
"""
|
||
|
||
from __future__ import annotations
|
||
|
||
import asyncio
|
||
import os
|
||
import subprocess
|
||
import sys
|
||
import tempfile
|
||
import time
|
||
import unicodedata
|
||
from pathlib import Path
|
||
|
||
import pytest
|
||
|
||
from raven.agent import workdir
|
||
from raven.agent.loop import AgentLoop
|
||
from raven.agent.loop.bundles import EngineWiring, ToolWiring, TurnPolicy
|
||
from raven.agent.loop.checkpoint import CheckpointService
|
||
from raven.config.raven import CheckpointConfig, RuntimeConfig
|
||
from raven.providers.base import LLMProvider, LLMResponse, ToolCallRequest
|
||
|
||
|
||
@pytest.fixture
|
||
def workspace():
|
||
with tempfile.TemporaryDirectory() as td:
|
||
yield Path(td)
|
||
|
||
|
||
async def _run_turn_body(agent: AgentLoop, workspace: Path):
|
||
"""Drive ``_run_agent_loop`` the way ``run_turn`` does: inside the
|
||
working-directory binding. The per-turn checkpoint reads that binding, so
|
||
an unbound call would snapshot nothing at all.
|
||
"""
|
||
with workdir.bind(workspace):
|
||
return await agent._run_agent_loop([{"role": "user", "content": "go"}])
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# CheckpointService unit
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
async def test_checkpoint_commits_then_noops_when_unchanged(workspace):
|
||
svc = CheckpointService(workspace)
|
||
(workspace / "a.py").write_text("print(1)\n", encoding="utf-8")
|
||
|
||
cid, changed = await svc.commit_turn("turn 1")
|
||
assert cid is not None
|
||
assert "a.py" in changed
|
||
|
||
# Nothing changed since the last snapshot → no new commit.
|
||
cid2, changed2 = await svc.commit_turn("turn 2")
|
||
assert cid2 is None
|
||
assert changed2 == []
|
||
|
||
|
||
async def test_checkpoint_does_not_touch_user_git(workspace):
|
||
"""When the workspace is itself a git repo, the shadow checkpoint must
|
||
not add commits to the user's real history (plan §7 fixture)."""
|
||
env = {"GIT_AUTHOR_NAME": "u", "GIT_AUTHOR_EMAIL": "u@u", "GIT_COMMITTER_NAME": "u", "GIT_COMMITTER_EMAIL": "u@u"}
|
||
subprocess.run(["git", "init", "-q"], cwd=workspace, check=True)
|
||
subprocess.run(
|
||
["git", "commit", "--allow-empty", "-qm", "base"], cwd=workspace, check=True, env={**_os_environ(), **env}
|
||
)
|
||
head_before = _git_head(workspace)
|
||
count_before = _git_count(workspace)
|
||
|
||
svc = CheckpointService(workspace)
|
||
(workspace / "b.py").write_text("x = 1\n", encoding="utf-8")
|
||
cid, changed = await svc.commit_turn("snap")
|
||
assert cid is not None
|
||
assert "b.py" in changed
|
||
|
||
assert _git_head(workspace) == head_before, "user HEAD must be unchanged"
|
||
assert _git_count(workspace) == count_before, "no commits added to user repo"
|
||
|
||
|
||
async def test_a_staged_tree_holds_what_a_file_said_before_it_changed(workspace):
|
||
"""What a command's diff is read against: the file as it stood when the tree
|
||
was staged, after the file has been rewritten on disk."""
|
||
svc = CheckpointService(workspace)
|
||
kept = workspace / "kept.txt"
|
||
kept.write_text("one\n", encoding="utf-8")
|
||
|
||
tree = await svc.stage_tree()
|
||
assert tree is not None
|
||
kept.write_text("two\n", encoding="utf-8")
|
||
(workspace / "new.txt").write_text("new\n", encoding="utf-8")
|
||
|
||
held = await svc.read_blobs(tree, [str(kept), str(workspace / "new.txt")], max_bytes=1024)
|
||
assert held == {str(kept): b"one\n"}, "a file the tree never had is unknown, not empty"
|
||
|
||
|
||
async def test_a_blob_is_left_out_past_the_cap_or_outside_the_work_tree(workspace, tmp_path_factory):
|
||
svc = CheckpointService(workspace)
|
||
big = workspace / "big.txt"
|
||
big.write_text("x" * 64, encoding="utf-8")
|
||
outside = tmp_path_factory.mktemp("outside") / "o.txt"
|
||
outside.write_text("o\n", encoding="utf-8")
|
||
|
||
tree = await svc.stage_tree()
|
||
assert tree is not None
|
||
|
||
assert await svc.read_blobs(tree, [str(big), str(outside)], max_bytes=63) == {}
|
||
assert await svc.read_blobs(tree, [str(big)], max_bytes=64) == {str(big): b"x" * 64}
|
||
|
||
|
||
async def test_staging_a_tree_leaves_the_turn_commit_its_own_changes(workspace):
|
||
"""The stage has an index of its own. Staged into the shared one, the turn's
|
||
commit would find this file already staged and report the turn as having
|
||
changed nothing -- the recovery prompt would lose it."""
|
||
svc = CheckpointService(workspace)
|
||
(workspace / "a.py").write_text("print(1)\n", encoding="utf-8")
|
||
|
||
assert await svc.stage_tree() is not None
|
||
cid, changed = await svc.commit_turn("turn 1")
|
||
|
||
assert cid is not None
|
||
assert changed == ["a.py"]
|
||
|
||
|
||
async def test_a_stale_staging_index_is_pruned_and_a_fresh_one_kept(workspace):
|
||
"""A process that died leaves its staging index behind. Pruned by age, never
|
||
by asking whether its pid is alive -- on Windows that question kills it."""
|
||
import os
|
||
import time
|
||
|
||
svc = CheckpointService(workspace)
|
||
assert await svc.stage_tree() is not None
|
||
git_dir = workspace / ".raven" / "shadow.git"
|
||
stale = git_dir / "exec-1.index"
|
||
fresh = git_dir / "exec-2.index"
|
||
stale.write_bytes(b"")
|
||
fresh.write_bytes(b"")
|
||
old = time.time() - 8 * 24 * 3600
|
||
os.utime(stale, (old, old))
|
||
|
||
again = CheckpointService(workspace)
|
||
assert await again.stage_tree() is not None
|
||
|
||
assert not stale.exists()
|
||
assert fresh.exists()
|
||
assert (git_dir / f"exec-{os.getpid()}.index").exists()
|
||
|
||
|
||
async def test_a_stage_that_timed_out_leaves_no_lock_behind(workspace, monkeypatch):
|
||
"""A killed ``git add`` leaves its ``index.lock``, and every later stage would
|
||
fail on it for the rest of the process -- one slow command would cost every
|
||
command after it its diff."""
|
||
import os
|
||
import subprocess
|
||
|
||
import raven.agent.loop.checkpoint as cp_module
|
||
|
||
svc = CheckpointService(workspace)
|
||
assert await svc.stage_tree() is not None
|
||
lock = workspace / ".raven" / "shadow.git" / f"exec-{os.getpid()}.index.lock"
|
||
real = subprocess.run
|
||
|
||
def _killed(cmd, **kwargs):
|
||
lock.write_bytes(b"")
|
||
raise subprocess.TimeoutExpired(cmd, 0.05)
|
||
|
||
monkeypatch.setattr(cp_module.subprocess, "run", _killed)
|
||
assert await svc.stage_tree() is None
|
||
assert not lock.exists()
|
||
|
||
monkeypatch.setattr(cp_module.subprocess, "run", real)
|
||
assert await svc.stage_tree() is not None
|
||
|
||
|
||
async def test_a_slow_first_stage_is_left_to_warm_the_index_behind_the_command(workspace, monkeypatch):
|
||
"""The first stage in a directory hashes every file and can take seconds.
|
||
Past the budget the call is told so, the stage keeps running, a command
|
||
that arrives meanwhile does not start a second one, and the one after it
|
||
finds the index warm."""
|
||
import asyncio
|
||
import subprocess
|
||
import threading
|
||
|
||
import raven.agent.loop.checkpoint as cp_module
|
||
|
||
svc = CheckpointService(workspace)
|
||
(workspace / "a.txt").write_text("a\n", encoding="utf-8")
|
||
real = subprocess.run
|
||
release = threading.Event()
|
||
adds: list[object] = []
|
||
|
||
def _slow(cmd, **kwargs):
|
||
if "add" in cmd:
|
||
adds.append(cmd)
|
||
release.wait(10)
|
||
return real(cmd, **kwargs)
|
||
|
||
monkeypatch.setattr(cp_module, "_STAGING", {})
|
||
monkeypatch.setattr(cp_module, "_STAGE_WAIT_SECONDS", 0.05)
|
||
monkeypatch.setattr(cp_module.subprocess, "run", _slow)
|
||
|
||
with pytest.raises(cp_module.StagingTimeoutError):
|
||
await svc.stage_tree()
|
||
with pytest.raises(cp_module.StagingTimeoutError):
|
||
await svc.stage_tree()
|
||
assert len(adds) == 1, "a stage still running is not started again"
|
||
|
||
release.set()
|
||
assert await asyncio.wrap_future(cp_module._STAGING[svc._stage_path()]) is not None
|
||
monkeypatch.setattr(cp_module, "_STAGE_WAIT_SECONDS", 30.0)
|
||
assert await svc.stage_tree() is not None
|
||
|
||
|
||
async def test_a_stage_waits_out_the_warm_up_and_stages_again(workspace, monkeypatch):
|
||
"""The warm-up started with the session may still be running when the first
|
||
command arrives. Its tree may predate what happened since, so it is waited
|
||
for and never used; the command stages its own after it."""
|
||
import subprocess
|
||
|
||
import raven.agent.loop.checkpoint as cp_module
|
||
|
||
svc = CheckpointService(workspace)
|
||
(workspace / "a.txt").write_text("a\n", encoding="utf-8")
|
||
real = subprocess.run
|
||
adds: list[object] = []
|
||
|
||
def _count(cmd, **kwargs):
|
||
if "add" in cmd:
|
||
adds.append(cmd)
|
||
if len(adds) == 1:
|
||
time.sleep(0.3)
|
||
return real(cmd, **kwargs)
|
||
|
||
monkeypatch.setattr(cp_module, "_STAGING", {})
|
||
monkeypatch.setattr(cp_module.subprocess, "run", _count)
|
||
|
||
await svc.warm()
|
||
(workspace / "a.txt").write_text("changed while warming\n", encoding="utf-8")
|
||
tree = await svc.stage_tree()
|
||
|
||
assert tree is not None
|
||
assert len(adds) == 2
|
||
held = await svc.read_blobs(tree, [str(workspace / "a.txt")], max_bytes=1024)
|
||
assert held == {str(workspace / "a.txt"): b"changed while warming\n"}
|
||
|
||
|
||
async def test_a_warm_up_already_running_is_not_started_again(workspace, monkeypatch):
|
||
"""Two turns starting in one directory warm it once: the second warm-up
|
||
would only queue a second hash of the same tree behind the first."""
|
||
import subprocess
|
||
|
||
import raven.agent.loop.checkpoint as cp_module
|
||
|
||
real = subprocess.run
|
||
adds: list[object] = []
|
||
|
||
def _count(cmd, **kwargs):
|
||
if "add" in cmd:
|
||
adds.append(cmd)
|
||
time.sleep(0.2)
|
||
return real(cmd, **kwargs)
|
||
|
||
monkeypatch.setattr(cp_module, "_STAGING", {})
|
||
monkeypatch.setattr(cp_module.subprocess, "run", _count)
|
||
svc = CheckpointService(workspace)
|
||
|
||
await svc.warm()
|
||
await CheckpointService(workspace).warm()
|
||
await asyncio.wrap_future(cp_module._STAGING[svc._stage_path()])
|
||
|
||
assert len(adds) == 1
|
||
|
||
|
||
async def test_a_warm_up_returns_before_the_repo_is_even_set_up(workspace, monkeypatch):
|
||
"""The turn awaits ``warm`` in front of its first model call, so everything
|
||
the warm-up does -- the repo's own ``git init`` and config as much as the
|
||
staging -- belongs on its thread. Awaited, a slow setup would be a first
|
||
reply that waits for it."""
|
||
import raven.agent.loop.checkpoint as cp_module
|
||
|
||
real = CheckpointService._ensure_init
|
||
|
||
async def _slow_init(self) -> bool:
|
||
await asyncio.sleep(1.0)
|
||
return await real(self)
|
||
|
||
monkeypatch.setattr(cp_module, "_STAGING", {})
|
||
monkeypatch.setattr(CheckpointService, "_ensure_init", _slow_init)
|
||
svc = CheckpointService(workspace)
|
||
|
||
started = time.monotonic()
|
||
await svc.warm()
|
||
assert time.monotonic() - started < 0.2
|
||
|
||
assert await asyncio.wrap_future(cp_module._STAGING[svc._stage_path()]) is not None
|
||
|
||
|
||
async def test_a_turn_that_ends_during_the_warm_up_still_commits(workspace, monkeypatch):
|
||
"""A turn can end before its warm-up has set the repo up. Two setups at once
|
||
fail on the config lock, and the one that lost was the turn's commit -- so
|
||
the commit waits for the warm-up's setup instead of racing it."""
|
||
import threading
|
||
|
||
import raven.agent.loop.checkpoint as cp_module
|
||
|
||
real = CheckpointService._init_repo
|
||
active = [0]
|
||
overlap = [0]
|
||
|
||
async def _tracked(self) -> bool:
|
||
active[0] += 1
|
||
overlap[0] = max(overlap[0], active[0])
|
||
try:
|
||
if threading.current_thread().name == "raven-stage":
|
||
await asyncio.sleep(0.3)
|
||
return await real(self)
|
||
finally:
|
||
active[0] -= 1
|
||
|
||
monkeypatch.setattr(cp_module, "_STAGING", {})
|
||
monkeypatch.setattr(CheckpointService, "_init_repo", _tracked)
|
||
(workspace / "a.py").write_text("print(1)\n", encoding="utf-8")
|
||
svc = CheckpointService(workspace)
|
||
|
||
await svc.warm()
|
||
cid, changed = await svc.commit_turn("turn 1")
|
||
|
||
assert cid is not None and changed == ["a.py"]
|
||
assert overlap[0] == 1, "the commit's setup ran beside the warm-up's"
|
||
|
||
|
||
async def test_a_directory_is_warmed_once_and_not_every_turn(workspace, monkeypatch):
|
||
"""Every turn starts with a warm-up call, and only the first does any work:
|
||
after it each command's own staging keeps the index warm, so another would
|
||
be one more stat walk of the whole tree per message for nothing."""
|
||
import subprocess
|
||
|
||
import raven.agent.loop.checkpoint as cp_module
|
||
|
||
real = subprocess.run
|
||
adds: list[object] = []
|
||
|
||
def _count(cmd, **kwargs):
|
||
if "add" in cmd:
|
||
adds.append(cmd)
|
||
return real(cmd, **kwargs)
|
||
|
||
monkeypatch.setattr(cp_module, "_STAGING", {})
|
||
monkeypatch.setattr(cp_module.subprocess, "run", _count)
|
||
svc = CheckpointService(workspace)
|
||
|
||
await svc.warm()
|
||
await asyncio.wrap_future(cp_module._STAGING[svc._stage_path()])
|
||
await svc.warm()
|
||
await svc.warm()
|
||
|
||
assert len(adds) == 1
|
||
|
||
|
||
async def test_two_commands_behind_one_warm_up_each_stage_their_own_tree(workspace, monkeypatch):
|
||
"""Two sessions in one directory both run a command while the warm-up is
|
||
still staging. Each waits it out and stages its own tree, one after the
|
||
other on the index's lock, rather than a second ``git add`` failing on the
|
||
lock beside the first."""
|
||
import asyncio
|
||
import subprocess
|
||
|
||
import raven.agent.loop.checkpoint as cp_module
|
||
|
||
svc = CheckpointService(workspace)
|
||
(workspace / "a.txt").write_text("a\n", encoding="utf-8")
|
||
real = subprocess.run
|
||
adds: list[object] = []
|
||
|
||
def _count(cmd, **kwargs):
|
||
if "add" in cmd:
|
||
adds.append(cmd)
|
||
if len(adds) == 1:
|
||
time.sleep(0.3)
|
||
return real(cmd, **kwargs)
|
||
|
||
monkeypatch.setattr(cp_module, "_STAGING", {})
|
||
monkeypatch.setattr(cp_module.subprocess, "run", _count)
|
||
|
||
await svc.warm()
|
||
first, second = await asyncio.gather(svc.stage_tree(), CheckpointService(workspace).stage_tree())
|
||
|
||
assert first is not None and second is not None
|
||
assert len(adds) == 3
|
||
|
||
|
||
async def test_a_slow_first_staging_is_not_cut_off_at_the_git_call_ceiling(workspace, monkeypatch):
|
||
"""Every command is held back until the staging finishes. Killed at the
|
||
ceiling a turn's own git calls use, a staging that needs longer is started
|
||
over, killed again, and never finishes -- so no command would ever run."""
|
||
import subprocess
|
||
|
||
import raven.agent.loop.checkpoint as cp_module
|
||
|
||
real = subprocess.run
|
||
ceilings: list[float] = []
|
||
|
||
def _spy(cmd, **kwargs):
|
||
if "add" in cmd:
|
||
ceilings.append(kwargs["timeout"])
|
||
return real(cmd, **kwargs)
|
||
|
||
monkeypatch.setattr(cp_module, "_STAGING", {})
|
||
monkeypatch.setattr(cp_module.subprocess, "run", _spy)
|
||
assert await CheckpointService(workspace).stage_tree() is not None
|
||
|
||
assert ceilings == [cp_module._STAGE_ADD_TIMEOUT_SECONDS]
|
||
assert ceilings[0] > cp_module._GIT_TIMEOUT_SECONDS * 10
|
||
|
||
|
||
async def test_a_command_behind_a_staging_past_the_budget_is_refused(workspace, monkeypatch):
|
||
"""The staging in front of the command -- here the warm-up -- has not
|
||
finished within the budget. The command is refused like one whose own
|
||
staging ran out of time, not run unmeasured because the staging in its way
|
||
was not its own."""
|
||
import subprocess
|
||
|
||
import raven.agent.loop.checkpoint as cp_module
|
||
|
||
real = subprocess.run
|
||
|
||
def _slow(cmd, **kwargs):
|
||
if "add" in cmd:
|
||
time.sleep(0.5)
|
||
return real(cmd, **kwargs)
|
||
|
||
monkeypatch.setattr(cp_module, "_STAGING", {})
|
||
monkeypatch.setattr(cp_module, "_STAGE_WAIT_SECONDS", 0.1)
|
||
monkeypatch.setattr(cp_module.subprocess, "run", _slow)
|
||
svc = CheckpointService(workspace)
|
||
|
||
await svc.warm()
|
||
warm_up = cp_module._STAGING[svc._stage_path()]
|
||
with pytest.raises(cp_module.StagingTimeoutError):
|
||
await svc.stage_tree()
|
||
|
||
# Giving up on it must not have cancelled it: its thread still finishes it.
|
||
assert await asyncio.wrap_future(warm_up) is not None
|
||
assert not warm_up.cancelled()
|
||
|
||
|
||
async def test_each_command_stages_the_directory_as_it_stands(workspace):
|
||
"""Every command stages afresh, so whatever changed the directory since
|
||
the last command -- a tool call, the user, another session -- is already in
|
||
the tree the next one is measured against."""
|
||
svc = CheckpointService(workspace)
|
||
kept = workspace / "a.txt"
|
||
kept.write_text("one\n", encoding="utf-8")
|
||
first = await svc.stage_tree()
|
||
|
||
kept.write_text("two\n", encoding="utf-8")
|
||
second = await svc.stage_tree()
|
||
|
||
assert first is not None and second is not None and second != first
|
||
assert await svc.read_blobs(second, [str(kept)], max_bytes=64) == {str(kept): b"two\n"}
|
||
|
||
|
||
@pytest.mark.production_timing # a first staging slower than any short wait is the property
|
||
async def test_a_slow_first_staging_is_waited_out_inside_the_command(workspace, monkeypatch):
|
||
"""The first staging of a large directory can take many seconds. The
|
||
command waits for it inside its own call -- the warm-up it finds running,
|
||
then its own staging -- and gets its tree, instead of failing and leaving
|
||
the model to retry."""
|
||
import subprocess
|
||
|
||
import raven.agent.loop.checkpoint as cp_module
|
||
|
||
real = subprocess.run
|
||
cold = [True]
|
||
|
||
def _cold_first(cmd, **kwargs):
|
||
if "add" in cmd and cold[0]:
|
||
cold[0] = False
|
||
time.sleep(1.0)
|
||
return real(cmd, **kwargs)
|
||
|
||
monkeypatch.setattr(cp_module, "_STAGING", {})
|
||
monkeypatch.setattr(cp_module.subprocess, "run", _cold_first)
|
||
(workspace / "a.txt").write_text("a\n", encoding="utf-8")
|
||
svc = CheckpointService(workspace)
|
||
|
||
await svc.warm()
|
||
started = time.monotonic()
|
||
tree = await svc.stage_tree()
|
||
|
||
assert tree is not None
|
||
assert time.monotonic() - started >= 0.5, "the command did not wait for the warm-up"
|
||
|
||
|
||
async def test_trackable_follows_the_repos_own_exclusion_rules(workspace, tmp_path_factory):
|
||
"""What a command's created files are checked against before their text
|
||
goes anywhere: the default excludes, the work-tree's .gitignore, and
|
||
nothing outside the work-tree."""
|
||
svc = CheckpointService(workspace)
|
||
(workspace / ".gitignore").write_text("build/\n", encoding="utf-8")
|
||
(workspace / "build").mkdir()
|
||
outside = tmp_path_factory.mktemp("outside") / "o.txt"
|
||
names = {
|
||
"plain": workspace / "notes.md",
|
||
"default exclude": workspace / ".env",
|
||
"key": workspace / "id_ed25519",
|
||
"user ignore": workspace / "build" / "out.js",
|
||
"outside": outside,
|
||
}
|
||
for path in names.values():
|
||
path.write_text("x\n", encoding="utf-8")
|
||
|
||
kept = await svc.trackable([str(path) for path in names.values()])
|
||
|
||
assert kept == {str(names["plain"])}
|
||
|
||
|
||
async def test_a_name_that_is_not_utf8_is_judged_and_looked_up_by_its_bytes(workspace):
|
||
"""A POSIX name holding byte 0xff reaches Python surrogate-escaped. The
|
||
query git reads must carry that byte, not fail to encode it after the
|
||
command has already run. Judged on the rules alone, so the names need not
|
||
exist (macOS refuses to create them)."""
|
||
svc = CheckpointService(workspace)
|
||
(workspace / ".gitignore").write_text("*.bin\n", encoding="utf-8")
|
||
plain = str(workspace / os.fsdecode(b"notes-\xff.md"))
|
||
ignored = str(workspace / os.fsdecode(b"blob-\xff.bin"))
|
||
|
||
assert await svc.trackable([plain, ignored]) == {plain}
|
||
|
||
tree = await svc.stage_tree()
|
||
assert tree is not None
|
||
assert await svc.read_blobs(tree, [plain], max_bytes=1024) == {}
|
||
|
||
|
||
async def test_trackable_asks_about_a_name_as_given_and_precomposed(workspace):
|
||
"""git on macOS precomposes a name it reads off the disk before matching an
|
||
ignore rule, but takes a path read from --stdin as written, so a file the
|
||
disk holds decomposed must still meet a rule written precomposed. The name
|
||
as given is asked about too: NFC also rewrites a compatibility ideograph,
|
||
which git leaves alone. Judged on the rules alone."""
|
||
svc = CheckpointService(workspace)
|
||
composed, hangul, ideograph = "secret-caf\u00e9.py", "\ud55c.py", "\uf900.py"
|
||
(workspace / ".gitignore").write_text(f"{composed}\n{hangul}\n{ideograph}\n", encoding="utf-8")
|
||
decomposed = [str(workspace / unicodedata.normalize("NFD", name)) for name in (composed, hangul)]
|
||
unruled = str(workspace / unicodedata.normalize("NFD", "notes-caf\u00e9.py"))
|
||
assert all(unicodedata.normalize("NFC", path) != path for path in [*decomposed, unruled])
|
||
assert unicodedata.normalize("NFC", ideograph) != ideograph
|
||
|
||
assert await svc.trackable([*decomposed, str(workspace / ideograph), unruled]) == {unruled}
|
||
|
||
|
||
async def test_a_decomposed_name_the_snapshot_leaves_out_is_not_trackable(workspace):
|
||
"""Held against the staged tree itself, in whichever spelling this host's
|
||
git matches its rules: with ``core.precomposeunicode`` on, which ``git init``
|
||
picks on macOS unless git's configuration already sets it, the precomposed
|
||
one, so a decomposed name under a precomposed rule is left out and must not
|
||
be vouched for. Elsewhere, or with the setting off, git stores that name,
|
||
and only the controls are left to check."""
|
||
svc = CheckpointService(workspace)
|
||
composed = "secret-caf\u00e9.py"
|
||
(workspace / ".gitignore").write_text(f"{composed}\n*.log\n", encoding="utf-8")
|
||
secret = unicodedata.normalize("NFD", composed)
|
||
unruled = unicodedata.normalize("NFD", "notes-caf\u00e9.py")
|
||
names = [secret, "build.log", unruled]
|
||
for name in names:
|
||
(workspace / name).write_text("x\n", encoding="utf-8")
|
||
|
||
tree = await svc.stage_tree()
|
||
assert tree is not None
|
||
_rc, out, _err = await svc._git("ls-tree", "-r", "-z", "--name-only", tree)
|
||
stored = {unicodedata.normalize("NFC", name) for name in out.split("\0") if name}
|
||
kept = await svc.trackable([str(workspace / name) for name in names])
|
||
|
||
left_out = [name for name in names if unicodedata.normalize("NFC", name) not in stored]
|
||
assert "build.log" in left_out and str(workspace / unruled) in kept
|
||
_rc, precompose, _err = await svc._git("config", "--bool", "core.precomposeunicode")
|
||
if sys.platform == "darwin" and precompose.strip() == "true":
|
||
assert secret in left_out
|
||
assert [name for name in left_out if str(workspace / name) in kept] == []
|
||
|
||
|
||
@pytest.mark.parametrize(
|
||
("name", "spelled"),
|
||
[
|
||
("cafe\u0301.py", "caf\u00e9.py"),
|
||
("\u1112\u1161\u11ab.py", "\ud55c.py"),
|
||
# Marks out of canonical order stay where they are; NFC gives U+1EC7.
|
||
("e\u0302\u0323.py", "\u00ea\u0323.py"),
|
||
# Nothing composes across a mark in between; NFC gives U+1EAC U+0316.
|
||
("A\u0323\u0316\u0302.py", "\u1ea0\u0316\u0302.py"),
|
||
# A character NFC replaces on its own is kept, here U+0341.
|
||
("=\u0338\u0341.py", "\u2260\u0341.py"),
|
||
# Compositions newer than Unicode 3.2 are not made, here Balinese U+1B06.
|
||
("\u1b05\u1b35.py", "\u1b05\u1b35.py"),
|
||
("\uf900.py", "\uf900.py"),
|
||
],
|
||
)
|
||
def test_git_s_macos_spelling_composes_neighbours_by_unicode_3_2(name, spelled):
|
||
"""Each spelling is what macOS's libiconv returned for the name, through
|
||
the UTF-8-MAC conversion git precomposes with."""
|
||
from raven.agent.loop.checkpoint import _precomposed
|
||
|
||
assert _precomposed(name) == spelled
|
||
|
||
|
||
@pytest.mark.parametrize(
|
||
("name", "rule"),
|
||
[
|
||
# How git matches off macOS, or with core.precomposeunicode off.
|
||
("secret-cafe\u0301.py", "secret-cafe\u0301.py"),
|
||
# How a rule is usually typed, though git on macOS spells this name otherwise.
|
||
("secret-e\u0302\u0323.py", "secret-\u1ec7.py"),
|
||
# How git on macOS spells it, which is neither.
|
||
("secret-e\u0302\u0323.py", "secret-\u00ea\u0323.py"),
|
||
],
|
||
ids=["as-given", "nfc", "git-on-macos"],
|
||
)
|
||
async def test_each_spelling_trackable_asks_about_withholds_a_name(workspace, name, rule):
|
||
"""Judged on the rules alone, so the names need not exist."""
|
||
svc = CheckpointService(workspace)
|
||
(workspace / ".gitignore").write_text(f"{rule}\n", encoding="utf-8")
|
||
unruled = str(workspace / "notes.py")
|
||
|
||
assert await svc.trackable([str(workspace / name), unruled]) == {unruled}
|
||
|
||
|
||
async def test_a_name_git_spells_as_neither_given_nor_nfc_is_judged_in_git_s_spelling(workspace):
|
||
"""A rule written as git on macOS spells a name that is neither the name as
|
||
given nor its NFC must still withhold it: judged on the rules alone, and
|
||
held against the staged tree where git precomposes."""
|
||
svc = CheckpointService(workspace)
|
||
reordered, crossed, unruled = "secret-e\u0302\u0323.py", "token-A\u0323\u0316\u0302.py", "notes-e\u0302\u0323.py"
|
||
rules = ["secret-\u00ea\u0323.py", "token-\u1ea0\u0316\u0302.py"]
|
||
assert all(
|
||
rule not in {name, unicodedata.normalize("NFC", name)} for rule, name in zip(rules, (reordered, crossed))
|
||
)
|
||
(workspace / ".gitignore").write_text("".join(f"{rule}\n" for rule in rules), encoding="utf-8")
|
||
names = [reordered, crossed, unruled]
|
||
for name in names:
|
||
(workspace / name).write_text("x\n", encoding="utf-8")
|
||
|
||
assert await svc.trackable([str(workspace / name) for name in names]) == {str(workspace / unruled)}
|
||
|
||
tree = await svc.stage_tree()
|
||
assert tree is not None
|
||
_rc, out, _err = await svc._git("ls-tree", "-r", "-z", "--name-only", tree)
|
||
stored = {unicodedata.normalize("NFD", name) for name in out.split("\0") if name}
|
||
assert unicodedata.normalize("NFD", unruled) in stored
|
||
_rc, precompose, _err = await svc._git("config", "--bool", "core.precomposeunicode")
|
||
if sys.platform == "darwin" and precompose.strip() == "true":
|
||
assert not {unicodedata.normalize("NFD", name) for name in (reordered, crossed)} & stored
|
||
|
||
|
||
async def test_trackable_vouches_for_nothing_when_git_cannot_answer(workspace, monkeypatch):
|
||
svc = CheckpointService(workspace)
|
||
(workspace / "notes.md").write_text("x\n", encoding="utf-8")
|
||
assert await svc.trackable([str(workspace / "notes.md")]) == {str(workspace / "notes.md")}
|
||
|
||
async def _broken(*_args, **_kwargs):
|
||
return 128, b"", b"fatal"
|
||
|
||
monkeypatch.setattr(svc, "_run", _broken)
|
||
assert await svc.trackable([str(workspace / "notes.md")]) == set()
|
||
|
||
|
||
async def test_a_first_stage_starts_from_the_last_turns_index(workspace, monkeypatch):
|
||
"""The turn's commit has already hashed the tree into the shared index, and a
|
||
staging index copied from it only has to stat what changed since. Started
|
||
empty, the first command in every process would pay for hashing the whole
|
||
tree again."""
|
||
import subprocess
|
||
|
||
import raven.agent.loop.checkpoint as cp_module
|
||
|
||
(workspace / "a.txt").write_text("a\n", encoding="utf-8")
|
||
assert (await CheckpointService(workspace).commit_turn("turn 1"))[0] is not None
|
||
shared = (workspace / ".raven" / "shadow.git" / "index").read_bytes()
|
||
real = subprocess.run
|
||
started_from: list[bytes] = []
|
||
|
||
def _spy(cmd, **kwargs):
|
||
if "add" in cmd:
|
||
started_from.append(Path(kwargs["env"]["GIT_INDEX_FILE"]).read_bytes())
|
||
return real(cmd, **kwargs)
|
||
|
||
monkeypatch.setattr(cp_module.subprocess, "run", _spy)
|
||
assert await CheckpointService(workspace).stage_tree() is not None
|
||
assert started_from == [shared]
|
||
|
||
|
||
def test_a_staging_still_running_does_not_hold_the_loop_open(workspace, monkeypatch):
|
||
"""Closing a loop cancels what is left on it. An asyncio subprocess still
|
||
starting at that moment never finishes cancelling (CPython 3.12, macOS) and
|
||
the close hangs -- which, for the gateway, is a shutdown that never ends."""
|
||
import asyncio
|
||
import subprocess
|
||
import threading
|
||
import time
|
||
|
||
import raven.agent.loop.checkpoint as cp_module
|
||
|
||
real = subprocess.run
|
||
release = threading.Event()
|
||
|
||
def _slow(cmd, **kwargs):
|
||
release.wait(10)
|
||
return real(cmd, **kwargs)
|
||
|
||
monkeypatch.setattr(cp_module, "_STAGING", {})
|
||
monkeypatch.setattr(cp_module, "_STAGE_WAIT_SECONDS", 0.05)
|
||
monkeypatch.setattr(cp_module.subprocess, "run", _slow)
|
||
|
||
async def one_call() -> bool:
|
||
try:
|
||
await CheckpointService(workspace).stage_tree()
|
||
except cp_module.StagingTimeoutError:
|
||
return True
|
||
return False
|
||
|
||
# On a thread of its own: asyncio.run on the main thread clears the default
|
||
# loop, and pytest-asyncio then makes one it never closes.
|
||
outcome: list[bool] = []
|
||
started = time.monotonic()
|
||
runner = threading.Thread(target=lambda: outcome.append(asyncio.run(one_call())))
|
||
runner.start()
|
||
runner.join(10)
|
||
assert outcome == [True]
|
||
assert time.monotonic() - started < 5
|
||
release.set()
|
||
|
||
|
||
def _os_environ():
|
||
import os
|
||
|
||
return dict(os.environ)
|
||
|
||
|
||
def _git_head(cwd: Path) -> str:
|
||
return subprocess.run(["git", "rev-parse", "HEAD"], cwd=cwd, capture_output=True, text=True).stdout.strip()
|
||
|
||
|
||
def _git_count(cwd: Path) -> str:
|
||
return subprocess.run(
|
||
["git", "rev-list", "--count", "HEAD"], cwd=cwd, capture_output=True, text=True
|
||
).stdout.strip()
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# Loop-level: max-iter interrupted vs baseline
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
class _ToolLoopProvider(LLMProvider):
|
||
"""Always asks to write the same file — never finishes, forcing max-iter."""
|
||
|
||
def __init__(self, path: str = "a.py") -> None:
|
||
super().__init__(api_key="test")
|
||
self._path = path
|
||
|
||
async def chat(
|
||
self,
|
||
messages,
|
||
tools=None,
|
||
model=None,
|
||
max_tokens=4096,
|
||
temperature=0.7,
|
||
reasoning_effort=None,
|
||
tool_choice=None,
|
||
):
|
||
return LLMResponse(
|
||
content="",
|
||
tool_calls=[
|
||
ToolCallRequest(
|
||
id="c1",
|
||
name="write_file",
|
||
arguments={"path": self._path, "content": "x = 1\n"},
|
||
)
|
||
],
|
||
finish_reason="tool_calls",
|
||
)
|
||
|
||
def get_default_model(self) -> str:
|
||
return "stub"
|
||
|
||
|
||
def _loop_agent(workspace: Path, *, checkpoint_enabled: bool) -> AgentLoop:
|
||
return AgentLoop(
|
||
provider=_ToolLoopProvider(),
|
||
workspace=workspace,
|
||
model="stub",
|
||
policy=TurnPolicy(max_iterations=2),
|
||
tools=ToolWiring(restrict_to_workspace=True),
|
||
engine=EngineWiring(
|
||
runtime_config=RuntimeConfig(
|
||
checkpoint=CheckpointConfig(
|
||
policy="always" if checkpoint_enabled else "never",
|
||
),
|
||
)
|
||
),
|
||
)
|
||
|
||
|
||
async def test_max_iter_interrupted_with_checkpoint(workspace):
|
||
agent = _loop_agent(workspace, checkpoint_enabled=True)
|
||
final, _used, _msgs, outcome = await _run_turn_body(agent, workspace)
|
||
assert outcome.status == "interrupted"
|
||
# Checkpoint-on no longer short-circuits to a fixed "iteration limit"
|
||
# notice: exhaustion always runs the synthesis wrap-up (here the stub
|
||
# cannot produce content, so it lands on the static fallback). The
|
||
# checkpoint metadata below — not the reply text — is what marks the turn
|
||
# interrupted and recoverable.
|
||
assert "maximum number of tool call iterations" in (final or "")
|
||
# The turn's edits were snapshotted and offered for recovery.
|
||
assert outcome.checkpoint_id is not None
|
||
assert "a.py" in outcome.edited_files
|
||
|
||
|
||
async def test_max_iter_baseline_preserved_when_disabled(workspace):
|
||
agent = _loop_agent(workspace, checkpoint_enabled=False)
|
||
final, _used, _msgs, outcome = await _run_turn_body(agent, workspace)
|
||
# Status is reported (harmless metadata) but behavior is baseline:
|
||
# original message text, no checkpoint.
|
||
assert outcome.status == "interrupted"
|
||
assert "maximum number of tool call iterations" in (final or "")
|
||
assert outcome.checkpoint_id is None
|
||
assert outcome.edited_files == []
|
||
|
||
|
||
# --- I3/I4: the soul cases — "half-done work must not pollute memory" --------
|
||
|
||
|
||
# Note: tests that spied on ``_trigger_local_extraction`` were removed when the
|
||
# embedded extraction path was retired by feature/integrate-everos (Phase B-1).
|
||
# The axiom "interrupted turn != completed turn" now lives in two places
|
||
# preserved by this merge:
|
||
# 1. Shadow-git snapshot is taken regardless (see test_max_iter_snapshot...)
|
||
# 2. ``outcome.status`` distinguishes interrupted vs completed for any caller
|
||
# that wants to gate downstream actions on it (the new after-turn
|
||
# pipeline at the caller level can choose to honor this — out of scope
|
||
# for the checkpoint itself).
|
||
|
||
|
||
# --- I5/I6: completed and error terminal states ------------------------------
|
||
|
||
|
||
class _WriteThenStopProvider(LLMProvider):
|
||
"""First turn writes a file, then stops — a normal completion that also
|
||
left an edit to snapshot."""
|
||
|
||
def __init__(self) -> None:
|
||
super().__init__(api_key="test")
|
||
self._n = 0
|
||
|
||
async def chat(
|
||
self,
|
||
messages,
|
||
tools=None,
|
||
model=None,
|
||
max_tokens=4096,
|
||
temperature=0.7,
|
||
reasoning_effort=None,
|
||
tool_choice=None,
|
||
):
|
||
self._n += 1
|
||
if self._n == 1:
|
||
return LLMResponse(
|
||
content="",
|
||
tool_calls=[
|
||
ToolCallRequest(
|
||
id="c1",
|
||
name="write_file",
|
||
arguments={"path": "done.py", "content": "ok\n"},
|
||
)
|
||
],
|
||
finish_reason="tool_calls",
|
||
)
|
||
return LLMResponse(content="all done", finish_reason="stop")
|
||
|
||
def get_default_model(self) -> str:
|
||
return "stub"
|
||
|
||
|
||
class _ErrorProvider(LLMProvider):
|
||
async def chat(
|
||
self,
|
||
messages,
|
||
tools=None,
|
||
model=None,
|
||
max_tokens=4096,
|
||
temperature=0.7,
|
||
reasoning_effort=None,
|
||
tool_choice=None,
|
||
):
|
||
return LLMResponse(content="fatal model error: bad request", finish_reason="error")
|
||
|
||
def get_default_model(self) -> str:
|
||
return "stub"
|
||
|
||
|
||
async def test_completed_status_and_snapshot(workspace):
|
||
agent = AgentLoop(
|
||
provider=_WriteThenStopProvider(),
|
||
workspace=workspace,
|
||
model="stub",
|
||
policy=TurnPolicy(max_iterations=5),
|
||
tools=ToolWiring(restrict_to_workspace=True),
|
||
engine=EngineWiring(runtime_config=RuntimeConfig(checkpoint=CheckpointConfig(policy="always"))),
|
||
)
|
||
final, _used, _msgs, outcome = await _run_turn_body(agent, workspace)
|
||
assert outcome.status == "completed"
|
||
assert "all done" in (final or "")
|
||
# The completed turn's edit was snapshotted...
|
||
assert outcome.checkpoint_id is not None
|
||
# ...but edited_files is only surfaced for interrupted turns (recovery).
|
||
assert outcome.edited_files == []
|
||
|
||
|
||
async def test_error_status(workspace):
|
||
agent = AgentLoop(
|
||
provider=_ErrorProvider(),
|
||
workspace=workspace,
|
||
model="stub",
|
||
policy=TurnPolicy(max_iterations=5),
|
||
tools=ToolWiring(restrict_to_workspace=True),
|
||
engine=EngineWiring(runtime_config=RuntimeConfig(checkpoint=CheckpointConfig(policy="always"))),
|
||
)
|
||
_final, _used, _msgs, outcome = await _run_turn_body(agent, workspace)
|
||
assert outcome.status == "error"
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# Recovery prompt injection
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_recovery_block_injected_into_next_user_message(workspace):
|
||
agent = _loop_agent(workspace, checkpoint_enabled=True)
|
||
agent._pending_recovery["sess"] = {"checkpoint_id": "abc123", "files": ["a.py", "b.py"]}
|
||
|
||
messages = [{"role": "user", "content": "continue please"}]
|
||
agent._inject_recovery_block("sess", messages)
|
||
|
||
injected = messages[-1]["content"]
|
||
assert "previous turn was interrupted" in injected.lower()
|
||
assert "a.py" in injected and "b.py" in injected
|
||
assert "abc123" in injected
|
||
assert "continue please" in injected
|
||
# Consumed exactly once.
|
||
assert "sess" not in agent._pending_recovery
|
||
|
||
|
||
def test_recovery_block_noop_without_pending(workspace):
|
||
agent = _loop_agent(workspace, checkpoint_enabled=True)
|
||
messages = [{"role": "user", "content": "hello"}]
|
||
agent._inject_recovery_block("sess", messages)
|
||
assert messages[-1]["content"] == "hello"
|
||
|
||
|
||
# --- U6/U8/U9: edge coverage -------------------------------------------------
|
||
|
||
|
||
async def test_checkpoint_captures_deletion(workspace):
|
||
"""add -A is edit-source-agnostic: a deletion (or any non-tool change) is
|
||
still snapshotted — the advantage over edit-tool-triggered approaches."""
|
||
svc = CheckpointService(workspace)
|
||
(workspace / "x.py").write_text("a\n", encoding="utf-8")
|
||
await svc.commit_turn("t1")
|
||
(workspace / "x.py").unlink()
|
||
cid, changed = await svc.commit_turn("t2")
|
||
assert cid is not None
|
||
assert "x.py" in changed
|
||
|
||
|
||
def test_recovery_block_injects_into_list_content(workspace):
|
||
"""Multimodal user message (content is a list) → recovery prepended as a
|
||
leading text block."""
|
||
agent = _loop_agent(workspace, checkpoint_enabled=True)
|
||
agent._pending_recovery["s"] = {"checkpoint_id": "cid9", "files": ["a.py"]}
|
||
messages = [
|
||
{
|
||
"role": "user",
|
||
"content": [
|
||
{"type": "text", "text": "hi"},
|
||
{"type": "image_url", "image_url": {"url": "data:..."}},
|
||
],
|
||
}
|
||
]
|
||
agent._inject_recovery_block("s", messages)
|
||
content = messages[-1]["content"]
|
||
assert isinstance(content, list)
|
||
assert content[0]["type"] == "text"
|
||
assert "interrupted" in content[0]["text"].lower()
|
||
# original blocks preserved after the injected one
|
||
assert content[1]["text"] == "hi"
|
||
|
||
|
||
def test_recovery_block_kept_when_last_not_user(workspace):
|
||
"""If the last message isn't the user turn, don't inject and don't lose the
|
||
pending recovery — keep it for the next (user-terminated) assembly."""
|
||
agent = _loop_agent(workspace, checkpoint_enabled=True)
|
||
agent._pending_recovery["s"] = {"checkpoint_id": "c", "files": ["a.py"]}
|
||
messages = [{"role": "assistant", "content": "x"}]
|
||
agent._inject_recovery_block("s", messages)
|
||
assert messages[-1]["content"] == "x"
|
||
assert "s" in agent._pending_recovery # not consumed
|
||
|
||
|
||
# --- E1/E2: cross-turn recovery through real assembly ------------------------
|
||
|
||
|
||
async def test_recovery_flows_interrupt_to_next_assembly(workspace):
|
||
"""End-to-end: an interrupted turn stashes recovery (as the caller does),
|
||
and the NEXT context assembly injects it into the user message — exercising
|
||
the real stash -> _assemble_context_messages -> inject wiring."""
|
||
agent = _loop_agent(workspace, checkpoint_enabled=True)
|
||
key = "tui:default"
|
||
|
||
# Turn 1 — force a max-iter interruption, then stash like the caller.
|
||
_f, _u, _m, outcome = await _run_turn_body(agent, workspace)
|
||
assert outcome.status == "interrupted"
|
||
with workdir.bind(workspace):
|
||
agent._stash_recovery(key, outcome)
|
||
assert key in agent._pending_recovery
|
||
|
||
# Turn 2 — real assembly must carry the recovery notice.
|
||
session = agent.sessions.get_or_create(key)
|
||
messages = await agent._assemble_context_messages(
|
||
session=session,
|
||
session_key=key,
|
||
current_message="continue",
|
||
)
|
||
last_user = messages[-1]
|
||
assert last_user["role"] == "user"
|
||
text = (
|
||
last_user["content"]
|
||
if isinstance(last_user["content"], str)
|
||
else next(b["text"] for b in last_user["content"] if b.get("type") == "text")
|
||
)
|
||
assert "interrupted" in text.lower()
|
||
assert "continue" in text
|
||
|
||
|
||
async def test_recovery_consumed_once(workspace):
|
||
"""A third assembly (after consumption) must not re-inject."""
|
||
agent = _loop_agent(workspace, checkpoint_enabled=True)
|
||
key = "tui:default"
|
||
agent._pending_recovery[key] = {"checkpoint_id": "c", "files": ["a.py"]}
|
||
session = agent.sessions.get_or_create(key)
|
||
|
||
m1 = await agent._assemble_context_messages(
|
||
session=session,
|
||
session_key=key,
|
||
current_message="first",
|
||
)
|
||
t1 = m1[-1]["content"]
|
||
assert "interrupted" in (t1 if isinstance(t1, str) else str(t1)).lower()
|
||
|
||
m2 = await agent._assemble_context_messages(
|
||
session=session,
|
||
session_key=key,
|
||
current_message="second",
|
||
)
|
||
t2 = m2[-1]["content"]
|
||
assert "interrupted" not in (t2 if isinstance(t2, str) else str(t2)).lower()
|
||
|
||
|
||
# --- F1: robustness — checkpoint must degrade, never crash the turn ----------
|
||
|
||
|
||
async def test_checkpoint_degrades_when_git_missing(workspace, monkeypatch):
|
||
"""If git is unavailable, commit_turn returns (None, []) and never raises —
|
||
the safety net must not break the turn it protects."""
|
||
import raven.agent.loop.checkpoint as ckpt_mod
|
||
|
||
async def _boom(*a, **k):
|
||
raise FileNotFoundError("git not found")
|
||
|
||
monkeypatch.setattr(ckpt_mod.asyncio, "create_subprocess_exec", _boom)
|
||
svc = ckpt_mod.CheckpointService(workspace)
|
||
(workspace / "a.py").write_text("x\n", encoding="utf-8")
|
||
cid, changed = await svc.commit_turn("t1")
|
||
assert cid is None
|
||
assert changed == []
|
||
|
||
|
||
async def test_loop_survives_checkpoint_failure(workspace, monkeypatch):
|
||
"""Same failure, but through the loop: the turn still returns a result."""
|
||
import raven.agent.loop.checkpoint as ckpt_mod
|
||
|
||
async def _boom(*a, **k):
|
||
raise OSError("disk gone")
|
||
|
||
monkeypatch.setattr(ckpt_mod.asyncio, "create_subprocess_exec", _boom)
|
||
agent = _loop_agent(workspace, checkpoint_enabled=True)
|
||
final, _u, _m, outcome = await _run_turn_body(agent, workspace)
|
||
assert outcome.status == "interrupted"
|
||
assert outcome.checkpoint_id is None # commit failed, but no crash
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# Scope of what gets snapshotted
|
||
#
|
||
# The working directory is the launch directory now, so `cd ~ && raven tui`
|
||
# would aim the shadow repo at a whole home. These pin both halves of the
|
||
# answer: refuse that root outright, and keep private keys out of the roots
|
||
# that are accepted.
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_checkpoint_refuses_a_home_or_wider_root(tmp_path, monkeypatch):
|
||
"""A home directory is not a workspace. `add -A` over one copies whatever
|
||
the user keeps there into a repo with history, every turn, and an
|
||
interrupted turn edits a project rather than a home -- so there is nothing
|
||
to recover in exchange."""
|
||
home = tmp_path / "home"
|
||
(home / "proj").mkdir(parents=True)
|
||
monkeypatch.setenv("HOME", str(home))
|
||
|
||
for refused in (home, tmp_path, Path(tmp_path.anchor)):
|
||
with pytest.raises(ValueError, match="home directory"):
|
||
CheckpointService(refused)
|
||
|
||
CheckpointService(home / "proj") # a project under home is still fine
|
||
|
||
|
||
async def test_checkpoint_excludes_private_keys(tmp_path, monkeypatch):
|
||
"""``*.key`` / ``*.pem`` never match ``id_ed25519``: private keys are
|
||
conventionally extensionless, so the credential patterns that look
|
||
exhaustive miss the most common secret on the machine."""
|
||
home = tmp_path / "home"
|
||
home.mkdir()
|
||
monkeypatch.setenv("HOME", str(home))
|
||
proj = home / "proj"
|
||
(proj / ".ssh").mkdir(parents=True)
|
||
(proj / ".ssh" / "id_ed25519").write_text("PRIVATE\n", encoding="utf-8")
|
||
(proj / "id_rsa").write_text("PRIVATE\n", encoding="utf-8")
|
||
(proj / "deploy.pem").write_text("PRIVATE\n", encoding="utf-8")
|
||
(proj / "main.py").write_text("print(1)\n", encoding="utf-8")
|
||
|
||
svc = CheckpointService(proj)
|
||
await svc.commit_turn("t1")
|
||
_rc, out, _err = await svc._git("ls-tree", "-r", "--name-only", "HEAD")
|
||
|
||
assert sorted(out.split()) == ["main.py"]
|
||
|
||
|
||
async def test_loop_runs_the_turn_when_the_root_is_refused(tmp_path, monkeypatch):
|
||
"""A refused root disables the safety net, it does not break the turn.
|
||
|
||
`raven tui` from a home directory is an ordinary thing to do, so the guard
|
||
has to degrade the way a bad `shadow_dir` already does -- one warning, and
|
||
the work still happens.
|
||
"""
|
||
home = tmp_path / "home"
|
||
home.mkdir()
|
||
monkeypatch.setenv("HOME", str(home))
|
||
(home / ".raven").mkdir()
|
||
(home / ".raven" / "config.json").write_text('{"permissions": {"mode": "full"}}')
|
||
|
||
agent = _loop_agent(home, checkpoint_enabled=True)
|
||
final, _used, _msgs, outcome = await _run_turn_body(agent, home)
|
||
|
||
assert outcome.status == "interrupted" # max-iter, as in the sibling tests
|
||
assert outcome.checkpoint_id is None # refused, so nothing was snapshotted
|
||
assert (home / "a.py").exists() # the turn's edit still landed
|
||
|
||
|
||
async def test_a_turn_cancelled_while_the_warm_up_sets_the_repo_up_does_not_orphan_its_staging(workspace, monkeypatch):
|
||
"""A turn that ends inside the warm-up's repo setup waits on it from its
|
||
commit, and a cancelled turn cancels that wait. Were the setup's own future
|
||
cancelled with it, the warm-up thread would fail on it before settling the
|
||
staging it had already published, and every later command in the directory
|
||
would wait out the full budget and be refused."""
|
||
import raven.agent.loop.checkpoint as cp_module
|
||
|
||
monkeypatch.setattr(cp_module, "_STAGING", {})
|
||
real_init = CheckpointService._init_repo
|
||
|
||
async def _slow_init(self: CheckpointService) -> bool:
|
||
await asyncio.sleep(0.3)
|
||
return await real_init(self)
|
||
|
||
monkeypatch.setattr(CheckpointService, "_init_repo", _slow_init)
|
||
(workspace / "a.txt").write_text("a\n", encoding="utf-8")
|
||
svc = CheckpointService(workspace)
|
||
|
||
await svc.warm()
|
||
waiter = asyncio.ensure_future(svc._ensure_init())
|
||
while not svc._initializing._done_callbacks:
|
||
await asyncio.sleep(0.01)
|
||
waiter.cancel()
|
||
with pytest.raises(asyncio.CancelledError):
|
||
await waiter
|
||
|
||
assert not svc._initializing.cancelled()
|
||
started = time.monotonic()
|
||
assert await svc.stage_tree() is not None
|
||
assert time.monotonic() - started < 30, "the staging was left for nobody to settle"
|
||
|
||
|
||
@pytest.mark.parametrize("broken", ["_stage_step", "_prepare_index"])
|
||
async def test_a_staging_that_fails_unexpectedly_still_answers(workspace, monkeypatch, broken):
|
||
"""A staging degrades to no tree, never to no answer: an unsettled one is
|
||
waited for until the budget runs out, and the command is refused for it."""
|
||
import raven.agent.loop.checkpoint as cp_module
|
||
|
||
monkeypatch.setattr(cp_module, "_STAGING", {})
|
||
|
||
def _raise(*_args: object, **_kwargs: object) -> None:
|
||
raise RuntimeError("unexpected")
|
||
|
||
svc = CheckpointService(workspace)
|
||
monkeypatch.setattr(svc, broken, _raise)
|
||
await svc.warm()
|
||
|
||
staging = cp_module._STAGING[svc._stage_path()]
|
||
assert await asyncio.wait_for(asyncio.wrap_future(staging), 30) is None
|
||
|
||
|
||
async def test_a_commands_own_staging_that_fails_unexpectedly_still_answers(workspace, monkeypatch):
|
||
"""The same guarantee on the path a command takes when nothing was warming."""
|
||
import raven.agent.loop.checkpoint as cp_module
|
||
|
||
monkeypatch.setattr(cp_module, "_STAGING", {})
|
||
|
||
def _raise(*_args: object, **_kwargs: object) -> None:
|
||
raise RuntimeError("unexpected")
|
||
|
||
svc = CheckpointService(workspace)
|
||
assert await svc.stage_tree() is not None
|
||
monkeypatch.setattr(svc, "_stage_step", _raise)
|
||
|
||
assert await asyncio.wait_for(svc.stage_tree(), 30) is None
|
||
|
||
|
||
async def test_a_git_call_whose_caller_stops_waiting_is_killed(workspace, monkeypatch):
|
||
"""A bounded measurement or a cancelled turn stops waiting on a git read;
|
||
the git itself must not be left running behind it."""
|
||
import os
|
||
|
||
svc = CheckpointService(workspace)
|
||
pid_file = workspace / "pid"
|
||
monkeypatch.setattr(
|
||
svc, "_command", lambda _args, _index: (["sh", "-c", f"echo $$ > {pid_file}; exec sleep 30"], None)
|
||
)
|
||
|
||
call = asyncio.ensure_future(svc._run(("cat-file", "--batch"), stdin=b""))
|
||
while not pid_file.exists() or not pid_file.read_text().strip():
|
||
await asyncio.sleep(0.01)
|
||
pid = int(pid_file.read_text())
|
||
call.cancel()
|
||
with pytest.raises(asyncio.CancelledError):
|
||
await call
|
||
|
||
for _ in range(200):
|
||
try:
|
||
os.kill(pid, 0)
|
||
except ProcessLookupError:
|
||
break
|
||
await asyncio.sleep(0.01)
|
||
else:
|
||
pytest.fail("the git process outlived the call that stopped waiting on it")
|