Raven-1/tests/test_runtime_checkpoint.py
Tong Li 0a1a711123
feat(*): add bounded Python diagnostics after code edits (#967)
## Summary

Raven-Code reports Python syntax errors after file edits but misses
unresolved names and equivalent feedback after shell edits. Add advisory
Pyflakes F821/F822/F823 checks to the existing tool result; optional
F401 notices remain off by default. No additional model call or code
execution is involved.

```text
write_file / edit_file
  -> inspect final source, compare with a baseline that can be read and analyzed
  -> retain syntax warnings; suppress existing name findings
exec
  -> require checkpoint visibility for the working directory
  -> read the changed files from the host's write record for the command
  -> ask visibility in batches of 256, select at most 8, check up to 4 concurrently
  -> append ordered diagnostics, preserving exit status and file metadata
```

Existing syntax failures remain visible after unrelated edits and
identical rewrites. A baseline that cannot be read, parsed or analyzed
is marked unknown; overwriting non-UTF-8 source still checks the known
replacement. Source nested too deeply for the parser or checker, and a
changed file that cannot be read, report diagnostics unavailable for
that file rather than passing as clean. Every check reads a file as the
bytes Python runs: text a write or edit stores is checked as its UTF-8
bytes, the content it replaced included, and a file from disk as its
bytes, each decoded as Python decodes source, by its coding declaration
or as UTF-8. Bytes that do not decode are reported as a syntax error at
the line where decoding stops, so a declaration the written text cannot
satisfy is reported however the file was changed. Noqa comments are read
as Ruff reads them, including after other comment directives and never
inside string literals: a file-level `ruff: noqa` / `flake8: noqa`
counts only on a line of its own, and a marker counts only when codes
follow its colon or, with no colon, whitespace, another comment or the
end of the comment follows it. Unlike Ruff, which takes the first
directive in a comment, every directive in it counts. The seeded tool
guide gives the syntax note's own advice. With no checkpoint or no
visibility verdict, exec diagnostics do not inspect changed file
contents; direct file checks remain available.

Pyflakes is imported when the tool face loads, not by whichever check
first needs it: imported lazily, its cold import fell inside that
check's one-second wait, and on a slow Windows runner the one test that
paid it reported diagnostics unavailable for a plain syntax error.
Private workers bound reads and checks to four active jobs per process.
Saturated capacity rejects new work without queuing, and
timeout/cancellation retains each slot until the actual work finishes.
Name checks are capped at 256 KiB of source and the syntax check at 2
MiB, and a check skipped for size says so in one line. Findings are
capped at eight per file, and exec diagnostic text at 6,000 characters
plus omission notices.

Runtime changes stay inside the Raven-Code code-flow plugin, except for
one fix to `CheckpointService.trackable`, the visibility verdict that
exec diagnostics and the host's exec write record both rely on. On
macOS, git precomposes a file name it reads off the disk before matching
an ignore rule, but takes the paths `trackable` sends to `git
check-ignore --stdin` as written. A file stored with a decomposed (NFD)
name under a `.gitignore` rule written precomposed (NFC) is therefore
left out of the checkpoint, yet main vouches for it, so the write
record's diff carries its text, and exec diagnostics would quote its
source lines to the model. Nor is git's precomposition NFC: it composes
only neighbouring characters, by Unicode 3.2's table, and reorders
nothing, so a name whose combining marks are out of canonical order
comes out as neither the name as given nor its NFC. `trackable` now asks
about each name as given, as NFC and in git's macOS spelling, and
withholds it when any spelling is ignored. Other shared
file/shell/snapshot implementations are unchanged.

Raven-Code has no separate dependency manifest, so Pyflakes uses the
existing root dependencies. It is the only added package; uv normalized
existing lockfile markers without changing other package versions.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- **Before:** on main, Raven-Code's file tools report Python syntax
errors only, and its exec reports nothing about the Python files a
command changed. On macOS, main's `CheckpointService.trackable` vouches
for a file stored as `secret_cafe` + U+0301 + `.py` under the rule
`secret_caf` + U+00E9 + `.py`: the user's own git ignores it and the
checkpoint's staged tree leaves it out, yet its text reaches the exec
write record's stored diff.
- **After:** on macOS 26.6.2 / arm64 with Apple Git 2.54.0, the command
below passes 364 tests on Python 3.12.13 and on 3.14.6. The full default
suite on 3.12.13 has 32283 passed, 169 skipped, 20 failed and 3 errors
over 22 test ids: 21 of them also fail on the exact base in the same
environment, and the other, a remote process-tree cancel test that
missed its deadline under the suite's load, passes when run alone. Tests
cover actual worker overlap, retained capacity after
timeout/cancellation, host-tool isolation, hidden-file quotas,
visibility batching through the host's write-record wiring, baseline
handling, parser-limit failures, the size bounds and their skip notes,
undecodable bytes placed where decoding stops, coding declarations in
written and edited text, the baseline read staying on a worker, the
checker loaded with its module, the file each unavailable note names,
comment directives and their marker boundary, the tool guide's agreement
with the syntax note, portable commands in paths with spaces/non-ASCII
characters, and the checkpoint's verdict on decomposed names and on
names git spells as neither given nor NFC, held against its own staged
tree. Removing any of the three spellings from the query fails the cases
that pin it, as does removing the marker boundary, the parser-limit
report, the unknown-baseline handling, the syntax check past the
name-check bound and the decoding report, checking written text as its
stored bytes, the worker around the baseline read, loading the checker
with its module, or the file name in an unavailable note; restoring the
guide's earlier advice fails its agreement test. git's macOS spelling is
what libiconv's UTF-8-MAC conversion returns, which git precomposes
with: composing neighbours by Unicode 3.2's table reproduced it for all
281,928 names built from every decomposable character's decomposition
(reordered, partly composed, or with a mark added), and across 784,220
further mixed names and pairs no name was spelled outside the three
asked; the same held under Python 3.13 and 3.14. The decomposed-name
check asserts the macOS outcome only where git precomposes names, so it
also passes with `core.precomposeunicode = false` set through
`GIT_CONFIG_GLOBAL`. Across 102 noqa comment variants the reported lines
match Ruff 0.15.12, except the three that hold several directives in one
comment. Through the real `AgentLoop`, code-flow tool face,
`CodeExecutor` and checkpoint, with the working directory under `/tmp`,
under `$TMPDIR` and in a path with spaces and CJK characters, exec
diagnostics report created, modified and decomposed-name files, follow
`working_dir`, report an undecodable file at the line where decoding
stops, a syntax error past 256 KiB and a skip past 2 MiB, and withhold
`.gitignore`d files, including decomposed names and names git spells as
neither given nor NFC; across repeated runs on both interpreters, a file
near the 256 KiB cap is checked in 52-135 ms (117-128 ms against a
baseline), the syntax of a file near 2 MiB in 318-371 ms, and 600
ignored files plus one visible file take 73-90 ms with the event loop
held at most 20 ms. At this head CI passes the unit suite on
Linux/Python 3.12 (32619 tests over four shards), and its two Windows
steps pass the diagnostic files on native Windows/Python 3.13: 122
file/worker tests, and 27 exec diagnostic tests with 17 pre-existing
executor tests deselected; the symlink fixture skips only where the host
cannot create links.

```sh
env -u SERPER_API_KEY -u RAVEN_MIRRORED_SERPER_API_KEY uv run --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_flow_exec.py tests/test_agents_code_flow_plugin.py tests/test_agents_code_flow_read_state.py tests/test_agents_code_launcher.py tests/test_workdir_snapshot.py tests/test_shell_command_writes.py tests/test_runtime_checkpoint.py tests/test_trajectory_upload.py::test_blocked_upload_hits_the_deadline -o addopts= -q --tb=short --maxfail=3
uv run --no-sync ruff check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_diagnostic_workers.py
uv run --no-sync ty check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py
uv run --no-sync lint-imports
uv run --no-sync ruff format --check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py
```

Native Windows CI commands:

```sh
uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py -q -n 0 --tb=short
uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_flow_exec.py -k test_exec_ -q -n 0 --tb=short
```

Ruff, format and type checks passed; all 10 import contracts kept; the
commit-message, large-file and source-language checks pass over the
branch. The launcher tests unset inherited search API keys to keep the
expected tool list deterministic.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Scope: Raven-Code's enabled code-flow tool face; installing Pyflakes
does not activate checks in other agents. The change is reversible and
never fixes/removes code or rolls back edits. Static diagnostics can
need judgment for dynamic names, target Python versions, and project
lint configuration; only explicit noqa directives are honored. Passing
diagnostics do not replace tests.

The `trackable` fix reaches every exec write record, not only
Raven-Code: a file whose name, as given, as NFC or in git's macOS
spelling, matches an ignore rule keeps its counts and loses its diff and
removal body. Where git matches another of those spellings, as off macOS
for a decomposed name under a precomposed rule, the checkpoint stores
the file but it is still withheld; its counts remain.

Exec diagnostics now require an active checkpoint. Noninteractive runs
using the default interactive checkpoint policy must explicitly enable
checkpoints to receive this feedback. Visibility is checked in batches
of up to 256 files and stops once eight visible files are found; each
batch holds the event loop while the host resolves its paths, about 130
ms for a full batch on native Windows. Each file-check wait is one
second, and post-command diagnostics have a three-second wait budget;
past it, or when they fail as a whole, they are dropped and the
command's result is kept. These are caller budgets, not hard I/O
deadlines: running OS reads and inherited synchronous host callbacks
cannot be forcibly interrupted. Private worker capacity remains bounded.
Between 256 KiB and 2 MiB only syntax is checked: near 2 MiB that took
0.3-0.4 s and about 190 MiB here, up to four checks run at once, and on
slower hardware such a check can outlast the one-second wait and report
diagnostics unavailable. Python's GIL means concurrency does not promise
a CPU speedup.

Windows CI exercises the diagnostics layer with the real native
DirectExecutor. The existing CodeExecutor environment probe uses POSIX
quoting and already fails on Windows at the base; this PR neither
changes that probe nor claims end-to-end compatibility for it. Symlink
setup is skipped only if the host cannot create the fixture.

Disable `pythonLint` for syntax-only feedback, `execPythonDiagnostics`
for no post-command checks, or `pythonSyntaxNote` for no diagnostics,
under `plugins.config["code-flow"].tools`. The PR can also be reverted
as a unit.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn>
Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-18 20:21:04 +08:00

1270 lines
48 KiB
Python
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

"""Per-turn shadow-git checkpoint + max-iter interrupted handling.
Covers the runtime-discipline safety net gated by
``config.runtime.checkpoint.policy`` × ``AgentLoop(interactive=...)``:
- CheckpointService snapshots the worktree without touching the user's .git.
- A max-iteration turn reports ``status="interrupted"`` (not a fake
completion) and the workspace is snapshotted for recovery.
- With policy="never" (or "interactive" + interactive=False), the loop is
behaviorally unchanged from baseline.
"""
from __future__ import annotations
import asyncio
import os
import subprocess
import sys
import tempfile
import time
import unicodedata
from pathlib import Path
import pytest
from raven.agent import workdir
from raven.agent.loop import AgentLoop
from raven.agent.loop.bundles import EngineWiring, ToolWiring, TurnPolicy
from raven.agent.loop.checkpoint import CheckpointService
from raven.config.raven import CheckpointConfig, RuntimeConfig
from raven.providers.base import LLMProvider, LLMResponse, ToolCallRequest
@pytest.fixture
def workspace():
with tempfile.TemporaryDirectory() as td:
yield Path(td)
async def _run_turn_body(agent: AgentLoop, workspace: Path):
"""Drive ``_run_agent_loop`` the way ``run_turn`` does: inside the
working-directory binding. The per-turn checkpoint reads that binding, so
an unbound call would snapshot nothing at all.
"""
with workdir.bind(workspace):
return await agent._run_agent_loop([{"role": "user", "content": "go"}])
# ---------------------------------------------------------------------------
# CheckpointService unit
# ---------------------------------------------------------------------------
async def test_checkpoint_commits_then_noops_when_unchanged(workspace):
svc = CheckpointService(workspace)
(workspace / "a.py").write_text("print(1)\n", encoding="utf-8")
cid, changed = await svc.commit_turn("turn 1")
assert cid is not None
assert "a.py" in changed
# Nothing changed since the last snapshot → no new commit.
cid2, changed2 = await svc.commit_turn("turn 2")
assert cid2 is None
assert changed2 == []
async def test_checkpoint_does_not_touch_user_git(workspace):
"""When the workspace is itself a git repo, the shadow checkpoint must
not add commits to the user's real history (plan §7 fixture)."""
env = {"GIT_AUTHOR_NAME": "u", "GIT_AUTHOR_EMAIL": "u@u", "GIT_COMMITTER_NAME": "u", "GIT_COMMITTER_EMAIL": "u@u"}
subprocess.run(["git", "init", "-q"], cwd=workspace, check=True)
subprocess.run(
["git", "commit", "--allow-empty", "-qm", "base"], cwd=workspace, check=True, env={**_os_environ(), **env}
)
head_before = _git_head(workspace)
count_before = _git_count(workspace)
svc = CheckpointService(workspace)
(workspace / "b.py").write_text("x = 1\n", encoding="utf-8")
cid, changed = await svc.commit_turn("snap")
assert cid is not None
assert "b.py" in changed
assert _git_head(workspace) == head_before, "user HEAD must be unchanged"
assert _git_count(workspace) == count_before, "no commits added to user repo"
async def test_a_staged_tree_holds_what_a_file_said_before_it_changed(workspace):
"""What a command's diff is read against: the file as it stood when the tree
was staged, after the file has been rewritten on disk."""
svc = CheckpointService(workspace)
kept = workspace / "kept.txt"
kept.write_text("one\n", encoding="utf-8")
tree = await svc.stage_tree()
assert tree is not None
kept.write_text("two\n", encoding="utf-8")
(workspace / "new.txt").write_text("new\n", encoding="utf-8")
held = await svc.read_blobs(tree, [str(kept), str(workspace / "new.txt")], max_bytes=1024)
assert held == {str(kept): b"one\n"}, "a file the tree never had is unknown, not empty"
async def test_a_blob_is_left_out_past_the_cap_or_outside_the_work_tree(workspace, tmp_path_factory):
svc = CheckpointService(workspace)
big = workspace / "big.txt"
big.write_text("x" * 64, encoding="utf-8")
outside = tmp_path_factory.mktemp("outside") / "o.txt"
outside.write_text("o\n", encoding="utf-8")
tree = await svc.stage_tree()
assert tree is not None
assert await svc.read_blobs(tree, [str(big), str(outside)], max_bytes=63) == {}
assert await svc.read_blobs(tree, [str(big)], max_bytes=64) == {str(big): b"x" * 64}
async def test_staging_a_tree_leaves_the_turn_commit_its_own_changes(workspace):
"""The stage has an index of its own. Staged into the shared one, the turn's
commit would find this file already staged and report the turn as having
changed nothing -- the recovery prompt would lose it."""
svc = CheckpointService(workspace)
(workspace / "a.py").write_text("print(1)\n", encoding="utf-8")
assert await svc.stage_tree() is not None
cid, changed = await svc.commit_turn("turn 1")
assert cid is not None
assert changed == ["a.py"]
async def test_a_stale_staging_index_is_pruned_and_a_fresh_one_kept(workspace):
"""A process that died leaves its staging index behind. Pruned by age, never
by asking whether its pid is alive -- on Windows that question kills it."""
import os
import time
svc = CheckpointService(workspace)
assert await svc.stage_tree() is not None
git_dir = workspace / ".raven" / "shadow.git"
stale = git_dir / "exec-1.index"
fresh = git_dir / "exec-2.index"
stale.write_bytes(b"")
fresh.write_bytes(b"")
old = time.time() - 8 * 24 * 3600
os.utime(stale, (old, old))
again = CheckpointService(workspace)
assert await again.stage_tree() is not None
assert not stale.exists()
assert fresh.exists()
assert (git_dir / f"exec-{os.getpid()}.index").exists()
async def test_a_stage_that_timed_out_leaves_no_lock_behind(workspace, monkeypatch):
"""A killed ``git add`` leaves its ``index.lock``, and every later stage would
fail on it for the rest of the process -- one slow command would cost every
command after it its diff."""
import os
import subprocess
import raven.agent.loop.checkpoint as cp_module
svc = CheckpointService(workspace)
assert await svc.stage_tree() is not None
lock = workspace / ".raven" / "shadow.git" / f"exec-{os.getpid()}.index.lock"
real = subprocess.run
def _killed(cmd, **kwargs):
lock.write_bytes(b"")
raise subprocess.TimeoutExpired(cmd, 0.05)
monkeypatch.setattr(cp_module.subprocess, "run", _killed)
assert await svc.stage_tree() is None
assert not lock.exists()
monkeypatch.setattr(cp_module.subprocess, "run", real)
assert await svc.stage_tree() is not None
async def test_a_slow_first_stage_is_left_to_warm_the_index_behind_the_command(workspace, monkeypatch):
"""The first stage in a directory hashes every file and can take seconds.
Past the budget the call is told so, the stage keeps running, a command
that arrives meanwhile does not start a second one, and the one after it
finds the index warm."""
import asyncio
import subprocess
import threading
import raven.agent.loop.checkpoint as cp_module
svc = CheckpointService(workspace)
(workspace / "a.txt").write_text("a\n", encoding="utf-8")
real = subprocess.run
release = threading.Event()
adds: list[object] = []
def _slow(cmd, **kwargs):
if "add" in cmd:
adds.append(cmd)
release.wait(10)
return real(cmd, **kwargs)
monkeypatch.setattr(cp_module, "_STAGING", {})
monkeypatch.setattr(cp_module, "_STAGE_WAIT_SECONDS", 0.05)
monkeypatch.setattr(cp_module.subprocess, "run", _slow)
with pytest.raises(cp_module.StagingTimeoutError):
await svc.stage_tree()
with pytest.raises(cp_module.StagingTimeoutError):
await svc.stage_tree()
assert len(adds) == 1, "a stage still running is not started again"
release.set()
assert await asyncio.wrap_future(cp_module._STAGING[svc._stage_path()]) is not None
monkeypatch.setattr(cp_module, "_STAGE_WAIT_SECONDS", 30.0)
assert await svc.stage_tree() is not None
async def test_a_stage_waits_out_the_warm_up_and_stages_again(workspace, monkeypatch):
"""The warm-up started with the session may still be running when the first
command arrives. Its tree may predate what happened since, so it is waited
for and never used; the command stages its own after it."""
import subprocess
import raven.agent.loop.checkpoint as cp_module
svc = CheckpointService(workspace)
(workspace / "a.txt").write_text("a\n", encoding="utf-8")
real = subprocess.run
adds: list[object] = []
def _count(cmd, **kwargs):
if "add" in cmd:
adds.append(cmd)
if len(adds) == 1:
time.sleep(0.3)
return real(cmd, **kwargs)
monkeypatch.setattr(cp_module, "_STAGING", {})
monkeypatch.setattr(cp_module.subprocess, "run", _count)
await svc.warm()
(workspace / "a.txt").write_text("changed while warming\n", encoding="utf-8")
tree = await svc.stage_tree()
assert tree is not None
assert len(adds) == 2
held = await svc.read_blobs(tree, [str(workspace / "a.txt")], max_bytes=1024)
assert held == {str(workspace / "a.txt"): b"changed while warming\n"}
async def test_a_warm_up_already_running_is_not_started_again(workspace, monkeypatch):
"""Two turns starting in one directory warm it once: the second warm-up
would only queue a second hash of the same tree behind the first."""
import subprocess
import raven.agent.loop.checkpoint as cp_module
real = subprocess.run
adds: list[object] = []
def _count(cmd, **kwargs):
if "add" in cmd:
adds.append(cmd)
time.sleep(0.2)
return real(cmd, **kwargs)
monkeypatch.setattr(cp_module, "_STAGING", {})
monkeypatch.setattr(cp_module.subprocess, "run", _count)
svc = CheckpointService(workspace)
await svc.warm()
await CheckpointService(workspace).warm()
await asyncio.wrap_future(cp_module._STAGING[svc._stage_path()])
assert len(adds) == 1
async def test_a_warm_up_returns_before_the_repo_is_even_set_up(workspace, monkeypatch):
"""The turn awaits ``warm`` in front of its first model call, so everything
the warm-up does -- the repo's own ``git init`` and config as much as the
staging -- belongs on its thread. Awaited, a slow setup would be a first
reply that waits for it."""
import raven.agent.loop.checkpoint as cp_module
real = CheckpointService._ensure_init
async def _slow_init(self) -> bool:
await asyncio.sleep(1.0)
return await real(self)
monkeypatch.setattr(cp_module, "_STAGING", {})
monkeypatch.setattr(CheckpointService, "_ensure_init", _slow_init)
svc = CheckpointService(workspace)
started = time.monotonic()
await svc.warm()
assert time.monotonic() - started < 0.2
assert await asyncio.wrap_future(cp_module._STAGING[svc._stage_path()]) is not None
async def test_a_turn_that_ends_during_the_warm_up_still_commits(workspace, monkeypatch):
"""A turn can end before its warm-up has set the repo up. Two setups at once
fail on the config lock, and the one that lost was the turn's commit -- so
the commit waits for the warm-up's setup instead of racing it."""
import threading
import raven.agent.loop.checkpoint as cp_module
real = CheckpointService._init_repo
active = [0]
overlap = [0]
async def _tracked(self) -> bool:
active[0] += 1
overlap[0] = max(overlap[0], active[0])
try:
if threading.current_thread().name == "raven-stage":
await asyncio.sleep(0.3)
return await real(self)
finally:
active[0] -= 1
monkeypatch.setattr(cp_module, "_STAGING", {})
monkeypatch.setattr(CheckpointService, "_init_repo", _tracked)
(workspace / "a.py").write_text("print(1)\n", encoding="utf-8")
svc = CheckpointService(workspace)
await svc.warm()
cid, changed = await svc.commit_turn("turn 1")
assert cid is not None and changed == ["a.py"]
assert overlap[0] == 1, "the commit's setup ran beside the warm-up's"
async def test_a_directory_is_warmed_once_and_not_every_turn(workspace, monkeypatch):
"""Every turn starts with a warm-up call, and only the first does any work:
after it each command's own staging keeps the index warm, so another would
be one more stat walk of the whole tree per message for nothing."""
import subprocess
import raven.agent.loop.checkpoint as cp_module
real = subprocess.run
adds: list[object] = []
def _count(cmd, **kwargs):
if "add" in cmd:
adds.append(cmd)
return real(cmd, **kwargs)
monkeypatch.setattr(cp_module, "_STAGING", {})
monkeypatch.setattr(cp_module.subprocess, "run", _count)
svc = CheckpointService(workspace)
await svc.warm()
await asyncio.wrap_future(cp_module._STAGING[svc._stage_path()])
await svc.warm()
await svc.warm()
assert len(adds) == 1
async def test_two_commands_behind_one_warm_up_each_stage_their_own_tree(workspace, monkeypatch):
"""Two sessions in one directory both run a command while the warm-up is
still staging. Each waits it out and stages its own tree, one after the
other on the index's lock, rather than a second ``git add`` failing on the
lock beside the first."""
import asyncio
import subprocess
import raven.agent.loop.checkpoint as cp_module
svc = CheckpointService(workspace)
(workspace / "a.txt").write_text("a\n", encoding="utf-8")
real = subprocess.run
adds: list[object] = []
def _count(cmd, **kwargs):
if "add" in cmd:
adds.append(cmd)
if len(adds) == 1:
time.sleep(0.3)
return real(cmd, **kwargs)
monkeypatch.setattr(cp_module, "_STAGING", {})
monkeypatch.setattr(cp_module.subprocess, "run", _count)
await svc.warm()
first, second = await asyncio.gather(svc.stage_tree(), CheckpointService(workspace).stage_tree())
assert first is not None and second is not None
assert len(adds) == 3
async def test_a_slow_first_staging_is_not_cut_off_at_the_git_call_ceiling(workspace, monkeypatch):
"""Every command is held back until the staging finishes. Killed at the
ceiling a turn's own git calls use, a staging that needs longer is started
over, killed again, and never finishes -- so no command would ever run."""
import subprocess
import raven.agent.loop.checkpoint as cp_module
real = subprocess.run
ceilings: list[float] = []
def _spy(cmd, **kwargs):
if "add" in cmd:
ceilings.append(kwargs["timeout"])
return real(cmd, **kwargs)
monkeypatch.setattr(cp_module, "_STAGING", {})
monkeypatch.setattr(cp_module.subprocess, "run", _spy)
assert await CheckpointService(workspace).stage_tree() is not None
assert ceilings == [cp_module._STAGE_ADD_TIMEOUT_SECONDS]
assert ceilings[0] > cp_module._GIT_TIMEOUT_SECONDS * 10
async def test_a_command_behind_a_staging_past_the_budget_is_refused(workspace, monkeypatch):
"""The staging in front of the command -- here the warm-up -- has not
finished within the budget. The command is refused like one whose own
staging ran out of time, not run unmeasured because the staging in its way
was not its own."""
import subprocess
import raven.agent.loop.checkpoint as cp_module
real = subprocess.run
def _slow(cmd, **kwargs):
if "add" in cmd:
time.sleep(0.5)
return real(cmd, **kwargs)
monkeypatch.setattr(cp_module, "_STAGING", {})
monkeypatch.setattr(cp_module, "_STAGE_WAIT_SECONDS", 0.1)
monkeypatch.setattr(cp_module.subprocess, "run", _slow)
svc = CheckpointService(workspace)
await svc.warm()
warm_up = cp_module._STAGING[svc._stage_path()]
with pytest.raises(cp_module.StagingTimeoutError):
await svc.stage_tree()
# Giving up on it must not have cancelled it: its thread still finishes it.
assert await asyncio.wrap_future(warm_up) is not None
assert not warm_up.cancelled()
async def test_each_command_stages_the_directory_as_it_stands(workspace):
"""Every command stages afresh, so whatever changed the directory since
the last command -- a tool call, the user, another session -- is already in
the tree the next one is measured against."""
svc = CheckpointService(workspace)
kept = workspace / "a.txt"
kept.write_text("one\n", encoding="utf-8")
first = await svc.stage_tree()
kept.write_text("two\n", encoding="utf-8")
second = await svc.stage_tree()
assert first is not None and second is not None and second != first
assert await svc.read_blobs(second, [str(kept)], max_bytes=64) == {str(kept): b"two\n"}
@pytest.mark.production_timing # a first staging slower than any short wait is the property
async def test_a_slow_first_staging_is_waited_out_inside_the_command(workspace, monkeypatch):
"""The first staging of a large directory can take many seconds. The
command waits for it inside its own call -- the warm-up it finds running,
then its own staging -- and gets its tree, instead of failing and leaving
the model to retry."""
import subprocess
import raven.agent.loop.checkpoint as cp_module
real = subprocess.run
cold = [True]
def _cold_first(cmd, **kwargs):
if "add" in cmd and cold[0]:
cold[0] = False
time.sleep(1.0)
return real(cmd, **kwargs)
monkeypatch.setattr(cp_module, "_STAGING", {})
monkeypatch.setattr(cp_module.subprocess, "run", _cold_first)
(workspace / "a.txt").write_text("a\n", encoding="utf-8")
svc = CheckpointService(workspace)
await svc.warm()
started = time.monotonic()
tree = await svc.stage_tree()
assert tree is not None
assert time.monotonic() - started >= 0.5, "the command did not wait for the warm-up"
async def test_trackable_follows_the_repos_own_exclusion_rules(workspace, tmp_path_factory):
"""What a command's created files are checked against before their text
goes anywhere: the default excludes, the work-tree's .gitignore, and
nothing outside the work-tree."""
svc = CheckpointService(workspace)
(workspace / ".gitignore").write_text("build/\n", encoding="utf-8")
(workspace / "build").mkdir()
outside = tmp_path_factory.mktemp("outside") / "o.txt"
names = {
"plain": workspace / "notes.md",
"default exclude": workspace / ".env",
"key": workspace / "id_ed25519",
"user ignore": workspace / "build" / "out.js",
"outside": outside,
}
for path in names.values():
path.write_text("x\n", encoding="utf-8")
kept = await svc.trackable([str(path) for path in names.values()])
assert kept == {str(names["plain"])}
async def test_a_name_that_is_not_utf8_is_judged_and_looked_up_by_its_bytes(workspace):
"""A POSIX name holding byte 0xff reaches Python surrogate-escaped. The
query git reads must carry that byte, not fail to encode it after the
command has already run. Judged on the rules alone, so the names need not
exist (macOS refuses to create them)."""
svc = CheckpointService(workspace)
(workspace / ".gitignore").write_text("*.bin\n", encoding="utf-8")
plain = str(workspace / os.fsdecode(b"notes-\xff.md"))
ignored = str(workspace / os.fsdecode(b"blob-\xff.bin"))
assert await svc.trackable([plain, ignored]) == {plain}
tree = await svc.stage_tree()
assert tree is not None
assert await svc.read_blobs(tree, [plain], max_bytes=1024) == {}
async def test_trackable_asks_about_a_name_as_given_and_precomposed(workspace):
"""git on macOS precomposes a name it reads off the disk before matching an
ignore rule, but takes a path read from --stdin as written, so a file the
disk holds decomposed must still meet a rule written precomposed. The name
as given is asked about too: NFC also rewrites a compatibility ideograph,
which git leaves alone. Judged on the rules alone."""
svc = CheckpointService(workspace)
composed, hangul, ideograph = "secret-caf\u00e9.py", "\ud55c.py", "\uf900.py"
(workspace / ".gitignore").write_text(f"{composed}\n{hangul}\n{ideograph}\n", encoding="utf-8")
decomposed = [str(workspace / unicodedata.normalize("NFD", name)) for name in (composed, hangul)]
unruled = str(workspace / unicodedata.normalize("NFD", "notes-caf\u00e9.py"))
assert all(unicodedata.normalize("NFC", path) != path for path in [*decomposed, unruled])
assert unicodedata.normalize("NFC", ideograph) != ideograph
assert await svc.trackable([*decomposed, str(workspace / ideograph), unruled]) == {unruled}
async def test_a_decomposed_name_the_snapshot_leaves_out_is_not_trackable(workspace):
"""Held against the staged tree itself, in whichever spelling this host's
git matches its rules: with ``core.precomposeunicode`` on, which ``git init``
picks on macOS unless git's configuration already sets it, the precomposed
one, so a decomposed name under a precomposed rule is left out and must not
be vouched for. Elsewhere, or with the setting off, git stores that name,
and only the controls are left to check."""
svc = CheckpointService(workspace)
composed = "secret-caf\u00e9.py"
(workspace / ".gitignore").write_text(f"{composed}\n*.log\n", encoding="utf-8")
secret = unicodedata.normalize("NFD", composed)
unruled = unicodedata.normalize("NFD", "notes-caf\u00e9.py")
names = [secret, "build.log", unruled]
for name in names:
(workspace / name).write_text("x\n", encoding="utf-8")
tree = await svc.stage_tree()
assert tree is not None
_rc, out, _err = await svc._git("ls-tree", "-r", "-z", "--name-only", tree)
stored = {unicodedata.normalize("NFC", name) for name in out.split("\0") if name}
kept = await svc.trackable([str(workspace / name) for name in names])
left_out = [name for name in names if unicodedata.normalize("NFC", name) not in stored]
assert "build.log" in left_out and str(workspace / unruled) in kept
_rc, precompose, _err = await svc._git("config", "--bool", "core.precomposeunicode")
if sys.platform == "darwin" and precompose.strip() == "true":
assert secret in left_out
assert [name for name in left_out if str(workspace / name) in kept] == []
@pytest.mark.parametrize(
("name", "spelled"),
[
("cafe\u0301.py", "caf\u00e9.py"),
("\u1112\u1161\u11ab.py", "\ud55c.py"),
# Marks out of canonical order stay where they are; NFC gives U+1EC7.
("e\u0302\u0323.py", "\u00ea\u0323.py"),
# Nothing composes across a mark in between; NFC gives U+1EAC U+0316.
("A\u0323\u0316\u0302.py", "\u1ea0\u0316\u0302.py"),
# A character NFC replaces on its own is kept, here U+0341.
("=\u0338\u0341.py", "\u2260\u0341.py"),
# Compositions newer than Unicode 3.2 are not made, here Balinese U+1B06.
("\u1b05\u1b35.py", "\u1b05\u1b35.py"),
("\uf900.py", "\uf900.py"),
],
)
def test_git_s_macos_spelling_composes_neighbours_by_unicode_3_2(name, spelled):
"""Each spelling is what macOS's libiconv returned for the name, through
the UTF-8-MAC conversion git precomposes with."""
from raven.agent.loop.checkpoint import _precomposed
assert _precomposed(name) == spelled
@pytest.mark.parametrize(
("name", "rule"),
[
# How git matches off macOS, or with core.precomposeunicode off.
("secret-cafe\u0301.py", "secret-cafe\u0301.py"),
# How a rule is usually typed, though git on macOS spells this name otherwise.
("secret-e\u0302\u0323.py", "secret-\u1ec7.py"),
# How git on macOS spells it, which is neither.
("secret-e\u0302\u0323.py", "secret-\u00ea\u0323.py"),
],
ids=["as-given", "nfc", "git-on-macos"],
)
async def test_each_spelling_trackable_asks_about_withholds_a_name(workspace, name, rule):
"""Judged on the rules alone, so the names need not exist."""
svc = CheckpointService(workspace)
(workspace / ".gitignore").write_text(f"{rule}\n", encoding="utf-8")
unruled = str(workspace / "notes.py")
assert await svc.trackable([str(workspace / name), unruled]) == {unruled}
async def test_a_name_git_spells_as_neither_given_nor_nfc_is_judged_in_git_s_spelling(workspace):
"""A rule written as git on macOS spells a name that is neither the name as
given nor its NFC must still withhold it: judged on the rules alone, and
held against the staged tree where git precomposes."""
svc = CheckpointService(workspace)
reordered, crossed, unruled = "secret-e\u0302\u0323.py", "token-A\u0323\u0316\u0302.py", "notes-e\u0302\u0323.py"
rules = ["secret-\u00ea\u0323.py", "token-\u1ea0\u0316\u0302.py"]
assert all(
rule not in {name, unicodedata.normalize("NFC", name)} for rule, name in zip(rules, (reordered, crossed))
)
(workspace / ".gitignore").write_text("".join(f"{rule}\n" for rule in rules), encoding="utf-8")
names = [reordered, crossed, unruled]
for name in names:
(workspace / name).write_text("x\n", encoding="utf-8")
assert await svc.trackable([str(workspace / name) for name in names]) == {str(workspace / unruled)}
tree = await svc.stage_tree()
assert tree is not None
_rc, out, _err = await svc._git("ls-tree", "-r", "-z", "--name-only", tree)
stored = {unicodedata.normalize("NFD", name) for name in out.split("\0") if name}
assert unicodedata.normalize("NFD", unruled) in stored
_rc, precompose, _err = await svc._git("config", "--bool", "core.precomposeunicode")
if sys.platform == "darwin" and precompose.strip() == "true":
assert not {unicodedata.normalize("NFD", name) for name in (reordered, crossed)} & stored
async def test_trackable_vouches_for_nothing_when_git_cannot_answer(workspace, monkeypatch):
svc = CheckpointService(workspace)
(workspace / "notes.md").write_text("x\n", encoding="utf-8")
assert await svc.trackable([str(workspace / "notes.md")]) == {str(workspace / "notes.md")}
async def _broken(*_args, **_kwargs):
return 128, b"", b"fatal"
monkeypatch.setattr(svc, "_run", _broken)
assert await svc.trackable([str(workspace / "notes.md")]) == set()
async def test_a_first_stage_starts_from_the_last_turns_index(workspace, monkeypatch):
"""The turn's commit has already hashed the tree into the shared index, and a
staging index copied from it only has to stat what changed since. Started
empty, the first command in every process would pay for hashing the whole
tree again."""
import subprocess
import raven.agent.loop.checkpoint as cp_module
(workspace / "a.txt").write_text("a\n", encoding="utf-8")
assert (await CheckpointService(workspace).commit_turn("turn 1"))[0] is not None
shared = (workspace / ".raven" / "shadow.git" / "index").read_bytes()
real = subprocess.run
started_from: list[bytes] = []
def _spy(cmd, **kwargs):
if "add" in cmd:
started_from.append(Path(kwargs["env"]["GIT_INDEX_FILE"]).read_bytes())
return real(cmd, **kwargs)
monkeypatch.setattr(cp_module.subprocess, "run", _spy)
assert await CheckpointService(workspace).stage_tree() is not None
assert started_from == [shared]
def test_a_staging_still_running_does_not_hold_the_loop_open(workspace, monkeypatch):
"""Closing a loop cancels what is left on it. An asyncio subprocess still
starting at that moment never finishes cancelling (CPython 3.12, macOS) and
the close hangs -- which, for the gateway, is a shutdown that never ends."""
import asyncio
import subprocess
import threading
import time
import raven.agent.loop.checkpoint as cp_module
real = subprocess.run
release = threading.Event()
def _slow(cmd, **kwargs):
release.wait(10)
return real(cmd, **kwargs)
monkeypatch.setattr(cp_module, "_STAGING", {})
monkeypatch.setattr(cp_module, "_STAGE_WAIT_SECONDS", 0.05)
monkeypatch.setattr(cp_module.subprocess, "run", _slow)
async def one_call() -> bool:
try:
await CheckpointService(workspace).stage_tree()
except cp_module.StagingTimeoutError:
return True
return False
# On a thread of its own: asyncio.run on the main thread clears the default
# loop, and pytest-asyncio then makes one it never closes.
outcome: list[bool] = []
started = time.monotonic()
runner = threading.Thread(target=lambda: outcome.append(asyncio.run(one_call())))
runner.start()
runner.join(10)
assert outcome == [True]
assert time.monotonic() - started < 5
release.set()
def _os_environ():
import os
return dict(os.environ)
def _git_head(cwd: Path) -> str:
return subprocess.run(["git", "rev-parse", "HEAD"], cwd=cwd, capture_output=True, text=True).stdout.strip()
def _git_count(cwd: Path) -> str:
return subprocess.run(
["git", "rev-list", "--count", "HEAD"], cwd=cwd, capture_output=True, text=True
).stdout.strip()
# ---------------------------------------------------------------------------
# Loop-level: max-iter interrupted vs baseline
# ---------------------------------------------------------------------------
class _ToolLoopProvider(LLMProvider):
"""Always asks to write the same file — never finishes, forcing max-iter."""
def __init__(self, path: str = "a.py") -> None:
super().__init__(api_key="test")
self._path = path
async def chat(
self,
messages,
tools=None,
model=None,
max_tokens=4096,
temperature=0.7,
reasoning_effort=None,
tool_choice=None,
):
return LLMResponse(
content="",
tool_calls=[
ToolCallRequest(
id="c1",
name="write_file",
arguments={"path": self._path, "content": "x = 1\n"},
)
],
finish_reason="tool_calls",
)
def get_default_model(self) -> str:
return "stub"
def _loop_agent(workspace: Path, *, checkpoint_enabled: bool) -> AgentLoop:
return AgentLoop(
provider=_ToolLoopProvider(),
workspace=workspace,
model="stub",
policy=TurnPolicy(max_iterations=2),
tools=ToolWiring(restrict_to_workspace=True),
engine=EngineWiring(
runtime_config=RuntimeConfig(
checkpoint=CheckpointConfig(
policy="always" if checkpoint_enabled else "never",
),
)
),
)
async def test_max_iter_interrupted_with_checkpoint(workspace):
agent = _loop_agent(workspace, checkpoint_enabled=True)
final, _used, _msgs, outcome = await _run_turn_body(agent, workspace)
assert outcome.status == "interrupted"
# Checkpoint-on no longer short-circuits to a fixed "iteration limit"
# notice: exhaustion always runs the synthesis wrap-up (here the stub
# cannot produce content, so it lands on the static fallback). The
# checkpoint metadata below — not the reply text — is what marks the turn
# interrupted and recoverable.
assert "maximum number of tool call iterations" in (final or "")
# The turn's edits were snapshotted and offered for recovery.
assert outcome.checkpoint_id is not None
assert "a.py" in outcome.edited_files
async def test_max_iter_baseline_preserved_when_disabled(workspace):
agent = _loop_agent(workspace, checkpoint_enabled=False)
final, _used, _msgs, outcome = await _run_turn_body(agent, workspace)
# Status is reported (harmless metadata) but behavior is baseline:
# original message text, no checkpoint.
assert outcome.status == "interrupted"
assert "maximum number of tool call iterations" in (final or "")
assert outcome.checkpoint_id is None
assert outcome.edited_files == []
# --- I3/I4: the soul cases — "half-done work must not pollute memory" --------
# Note: tests that spied on ``_trigger_local_extraction`` were removed when the
# embedded extraction path was retired by feature/integrate-everos (Phase B-1).
# The axiom "interrupted turn != completed turn" now lives in two places
# preserved by this merge:
# 1. Shadow-git snapshot is taken regardless (see test_max_iter_snapshot...)
# 2. ``outcome.status`` distinguishes interrupted vs completed for any caller
# that wants to gate downstream actions on it (the new after-turn
# pipeline at the caller level can choose to honor this — out of scope
# for the checkpoint itself).
# --- I5/I6: completed and error terminal states ------------------------------
class _WriteThenStopProvider(LLMProvider):
"""First turn writes a file, then stops — a normal completion that also
left an edit to snapshot."""
def __init__(self) -> None:
super().__init__(api_key="test")
self._n = 0
async def chat(
self,
messages,
tools=None,
model=None,
max_tokens=4096,
temperature=0.7,
reasoning_effort=None,
tool_choice=None,
):
self._n += 1
if self._n == 1:
return LLMResponse(
content="",
tool_calls=[
ToolCallRequest(
id="c1",
name="write_file",
arguments={"path": "done.py", "content": "ok\n"},
)
],
finish_reason="tool_calls",
)
return LLMResponse(content="all done", finish_reason="stop")
def get_default_model(self) -> str:
return "stub"
class _ErrorProvider(LLMProvider):
async def chat(
self,
messages,
tools=None,
model=None,
max_tokens=4096,
temperature=0.7,
reasoning_effort=None,
tool_choice=None,
):
return LLMResponse(content="fatal model error: bad request", finish_reason="error")
def get_default_model(self) -> str:
return "stub"
async def test_completed_status_and_snapshot(workspace):
agent = AgentLoop(
provider=_WriteThenStopProvider(),
workspace=workspace,
model="stub",
policy=TurnPolicy(max_iterations=5),
tools=ToolWiring(restrict_to_workspace=True),
engine=EngineWiring(runtime_config=RuntimeConfig(checkpoint=CheckpointConfig(policy="always"))),
)
final, _used, _msgs, outcome = await _run_turn_body(agent, workspace)
assert outcome.status == "completed"
assert "all done" in (final or "")
# The completed turn's edit was snapshotted...
assert outcome.checkpoint_id is not None
# ...but edited_files is only surfaced for interrupted turns (recovery).
assert outcome.edited_files == []
async def test_error_status(workspace):
agent = AgentLoop(
provider=_ErrorProvider(),
workspace=workspace,
model="stub",
policy=TurnPolicy(max_iterations=5),
tools=ToolWiring(restrict_to_workspace=True),
engine=EngineWiring(runtime_config=RuntimeConfig(checkpoint=CheckpointConfig(policy="always"))),
)
_final, _used, _msgs, outcome = await _run_turn_body(agent, workspace)
assert outcome.status == "error"
# ---------------------------------------------------------------------------
# Recovery prompt injection
# ---------------------------------------------------------------------------
def test_recovery_block_injected_into_next_user_message(workspace):
agent = _loop_agent(workspace, checkpoint_enabled=True)
agent._pending_recovery["sess"] = {"checkpoint_id": "abc123", "files": ["a.py", "b.py"]}
messages = [{"role": "user", "content": "continue please"}]
agent._inject_recovery_block("sess", messages)
injected = messages[-1]["content"]
assert "previous turn was interrupted" in injected.lower()
assert "a.py" in injected and "b.py" in injected
assert "abc123" in injected
assert "continue please" in injected
# Consumed exactly once.
assert "sess" not in agent._pending_recovery
def test_recovery_block_noop_without_pending(workspace):
agent = _loop_agent(workspace, checkpoint_enabled=True)
messages = [{"role": "user", "content": "hello"}]
agent._inject_recovery_block("sess", messages)
assert messages[-1]["content"] == "hello"
# --- U6/U8/U9: edge coverage -------------------------------------------------
async def test_checkpoint_captures_deletion(workspace):
"""add -A is edit-source-agnostic: a deletion (or any non-tool change) is
still snapshotted — the advantage over edit-tool-triggered approaches."""
svc = CheckpointService(workspace)
(workspace / "x.py").write_text("a\n", encoding="utf-8")
await svc.commit_turn("t1")
(workspace / "x.py").unlink()
cid, changed = await svc.commit_turn("t2")
assert cid is not None
assert "x.py" in changed
def test_recovery_block_injects_into_list_content(workspace):
"""Multimodal user message (content is a list) → recovery prepended as a
leading text block."""
agent = _loop_agent(workspace, checkpoint_enabled=True)
agent._pending_recovery["s"] = {"checkpoint_id": "cid9", "files": ["a.py"]}
messages = [
{
"role": "user",
"content": [
{"type": "text", "text": "hi"},
{"type": "image_url", "image_url": {"url": "data:..."}},
],
}
]
agent._inject_recovery_block("s", messages)
content = messages[-1]["content"]
assert isinstance(content, list)
assert content[0]["type"] == "text"
assert "interrupted" in content[0]["text"].lower()
# original blocks preserved after the injected one
assert content[1]["text"] == "hi"
def test_recovery_block_kept_when_last_not_user(workspace):
"""If the last message isn't the user turn, don't inject and don't lose the
pending recovery — keep it for the next (user-terminated) assembly."""
agent = _loop_agent(workspace, checkpoint_enabled=True)
agent._pending_recovery["s"] = {"checkpoint_id": "c", "files": ["a.py"]}
messages = [{"role": "assistant", "content": "x"}]
agent._inject_recovery_block("s", messages)
assert messages[-1]["content"] == "x"
assert "s" in agent._pending_recovery # not consumed
# --- E1/E2: cross-turn recovery through real assembly ------------------------
async def test_recovery_flows_interrupt_to_next_assembly(workspace):
"""End-to-end: an interrupted turn stashes recovery (as the caller does),
and the NEXT context assembly injects it into the user message — exercising
the real stash -> _assemble_context_messages -> inject wiring."""
agent = _loop_agent(workspace, checkpoint_enabled=True)
key = "tui:default"
# Turn 1 — force a max-iter interruption, then stash like the caller.
_f, _u, _m, outcome = await _run_turn_body(agent, workspace)
assert outcome.status == "interrupted"
with workdir.bind(workspace):
agent._stash_recovery(key, outcome)
assert key in agent._pending_recovery
# Turn 2 — real assembly must carry the recovery notice.
session = agent.sessions.get_or_create(key)
messages = await agent._assemble_context_messages(
session=session,
session_key=key,
current_message="continue",
)
last_user = messages[-1]
assert last_user["role"] == "user"
text = (
last_user["content"]
if isinstance(last_user["content"], str)
else next(b["text"] for b in last_user["content"] if b.get("type") == "text")
)
assert "interrupted" in text.lower()
assert "continue" in text
async def test_recovery_consumed_once(workspace):
"""A third assembly (after consumption) must not re-inject."""
agent = _loop_agent(workspace, checkpoint_enabled=True)
key = "tui:default"
agent._pending_recovery[key] = {"checkpoint_id": "c", "files": ["a.py"]}
session = agent.sessions.get_or_create(key)
m1 = await agent._assemble_context_messages(
session=session,
session_key=key,
current_message="first",
)
t1 = m1[-1]["content"]
assert "interrupted" in (t1 if isinstance(t1, str) else str(t1)).lower()
m2 = await agent._assemble_context_messages(
session=session,
session_key=key,
current_message="second",
)
t2 = m2[-1]["content"]
assert "interrupted" not in (t2 if isinstance(t2, str) else str(t2)).lower()
# --- F1: robustness — checkpoint must degrade, never crash the turn ----------
async def test_checkpoint_degrades_when_git_missing(workspace, monkeypatch):
"""If git is unavailable, commit_turn returns (None, []) and never raises —
the safety net must not break the turn it protects."""
import raven.agent.loop.checkpoint as ckpt_mod
async def _boom(*a, **k):
raise FileNotFoundError("git not found")
monkeypatch.setattr(ckpt_mod.asyncio, "create_subprocess_exec", _boom)
svc = ckpt_mod.CheckpointService(workspace)
(workspace / "a.py").write_text("x\n", encoding="utf-8")
cid, changed = await svc.commit_turn("t1")
assert cid is None
assert changed == []
async def test_loop_survives_checkpoint_failure(workspace, monkeypatch):
"""Same failure, but through the loop: the turn still returns a result."""
import raven.agent.loop.checkpoint as ckpt_mod
async def _boom(*a, **k):
raise OSError("disk gone")
monkeypatch.setattr(ckpt_mod.asyncio, "create_subprocess_exec", _boom)
agent = _loop_agent(workspace, checkpoint_enabled=True)
final, _u, _m, outcome = await _run_turn_body(agent, workspace)
assert outcome.status == "interrupted"
assert outcome.checkpoint_id is None # commit failed, but no crash
# ---------------------------------------------------------------------------
# Scope of what gets snapshotted
#
# The working directory is the launch directory now, so `cd ~ && raven tui`
# would aim the shadow repo at a whole home. These pin both halves of the
# answer: refuse that root outright, and keep private keys out of the roots
# that are accepted.
# ---------------------------------------------------------------------------
def test_checkpoint_refuses_a_home_or_wider_root(tmp_path, monkeypatch):
"""A home directory is not a workspace. `add -A` over one copies whatever
the user keeps there into a repo with history, every turn, and an
interrupted turn edits a project rather than a home -- so there is nothing
to recover in exchange."""
home = tmp_path / "home"
(home / "proj").mkdir(parents=True)
monkeypatch.setenv("HOME", str(home))
for refused in (home, tmp_path, Path(tmp_path.anchor)):
with pytest.raises(ValueError, match="home directory"):
CheckpointService(refused)
CheckpointService(home / "proj") # a project under home is still fine
async def test_checkpoint_excludes_private_keys(tmp_path, monkeypatch):
"""``*.key`` / ``*.pem`` never match ``id_ed25519``: private keys are
conventionally extensionless, so the credential patterns that look
exhaustive miss the most common secret on the machine."""
home = tmp_path / "home"
home.mkdir()
monkeypatch.setenv("HOME", str(home))
proj = home / "proj"
(proj / ".ssh").mkdir(parents=True)
(proj / ".ssh" / "id_ed25519").write_text("PRIVATE\n", encoding="utf-8")
(proj / "id_rsa").write_text("PRIVATE\n", encoding="utf-8")
(proj / "deploy.pem").write_text("PRIVATE\n", encoding="utf-8")
(proj / "main.py").write_text("print(1)\n", encoding="utf-8")
svc = CheckpointService(proj)
await svc.commit_turn("t1")
_rc, out, _err = await svc._git("ls-tree", "-r", "--name-only", "HEAD")
assert sorted(out.split()) == ["main.py"]
async def test_loop_runs_the_turn_when_the_root_is_refused(tmp_path, monkeypatch):
"""A refused root disables the safety net, it does not break the turn.
`raven tui` from a home directory is an ordinary thing to do, so the guard
has to degrade the way a bad `shadow_dir` already does -- one warning, and
the work still happens.
"""
home = tmp_path / "home"
home.mkdir()
monkeypatch.setenv("HOME", str(home))
(home / ".raven").mkdir()
(home / ".raven" / "config.json").write_text('{"permissions": {"mode": "full"}}')
agent = _loop_agent(home, checkpoint_enabled=True)
final, _used, _msgs, outcome = await _run_turn_body(agent, home)
assert outcome.status == "interrupted" # max-iter, as in the sibling tests
assert outcome.checkpoint_id is None # refused, so nothing was snapshotted
assert (home / "a.py").exists() # the turn's edit still landed
async def test_a_turn_cancelled_while_the_warm_up_sets_the_repo_up_does_not_orphan_its_staging(workspace, monkeypatch):
"""A turn that ends inside the warm-up's repo setup waits on it from its
commit, and a cancelled turn cancels that wait. Were the setup's own future
cancelled with it, the warm-up thread would fail on it before settling the
staging it had already published, and every later command in the directory
would wait out the full budget and be refused."""
import raven.agent.loop.checkpoint as cp_module
monkeypatch.setattr(cp_module, "_STAGING", {})
real_init = CheckpointService._init_repo
async def _slow_init(self: CheckpointService) -> bool:
await asyncio.sleep(0.3)
return await real_init(self)
monkeypatch.setattr(CheckpointService, "_init_repo", _slow_init)
(workspace / "a.txt").write_text("a\n", encoding="utf-8")
svc = CheckpointService(workspace)
await svc.warm()
waiter = asyncio.ensure_future(svc._ensure_init())
while not svc._initializing._done_callbacks:
await asyncio.sleep(0.01)
waiter.cancel()
with pytest.raises(asyncio.CancelledError):
await waiter
assert not svc._initializing.cancelled()
started = time.monotonic()
assert await svc.stage_tree() is not None
assert time.monotonic() - started < 30, "the staging was left for nobody to settle"
@pytest.mark.parametrize("broken", ["_stage_step", "_prepare_index"])
async def test_a_staging_that_fails_unexpectedly_still_answers(workspace, monkeypatch, broken):
"""A staging degrades to no tree, never to no answer: an unsettled one is
waited for until the budget runs out, and the command is refused for it."""
import raven.agent.loop.checkpoint as cp_module
monkeypatch.setattr(cp_module, "_STAGING", {})
def _raise(*_args: object, **_kwargs: object) -> None:
raise RuntimeError("unexpected")
svc = CheckpointService(workspace)
monkeypatch.setattr(svc, broken, _raise)
await svc.warm()
staging = cp_module._STAGING[svc._stage_path()]
assert await asyncio.wait_for(asyncio.wrap_future(staging), 30) is None
async def test_a_commands_own_staging_that_fails_unexpectedly_still_answers(workspace, monkeypatch):
"""The same guarantee on the path a command takes when nothing was warming."""
import raven.agent.loop.checkpoint as cp_module
monkeypatch.setattr(cp_module, "_STAGING", {})
def _raise(*_args: object, **_kwargs: object) -> None:
raise RuntimeError("unexpected")
svc = CheckpointService(workspace)
assert await svc.stage_tree() is not None
monkeypatch.setattr(svc, "_stage_step", _raise)
assert await asyncio.wait_for(svc.stage_tree(), 30) is None
async def test_a_git_call_whose_caller_stops_waiting_is_killed(workspace, monkeypatch):
"""A bounded measurement or a cancelled turn stops waiting on a git read;
the git itself must not be left running behind it."""
import os
svc = CheckpointService(workspace)
pid_file = workspace / "pid"
monkeypatch.setattr(
svc, "_command", lambda _args, _index: (["sh", "-c", f"echo $$ > {pid_file}; exec sleep 30"], None)
)
call = asyncio.ensure_future(svc._run(("cat-file", "--batch"), stdin=b""))
while not pid_file.exists() or not pid_file.read_text().strip():
await asyncio.sleep(0.01)
pid = int(pid_file.read_text())
call.cancel()
with pytest.raises(asyncio.CancelledError):
await call
for _ in range(200):
try:
os.kill(pid, 0)
except ProcessLookupError:
break
await asyncio.sleep(0.01)
else:
pytest.fail("the git process outlived the call that stopped waiting on it")