## Summary Raven-Code reports Python syntax errors after file edits but misses unresolved names and equivalent feedback after shell edits. Add advisory Pyflakes F821/F822/F823 checks to the existing tool result; optional F401 notices remain off by default. No additional model call or code execution is involved. ```text write_file / edit_file -> inspect final source, compare with a baseline that can be read and analyzed -> retain syntax warnings; suppress existing name findings exec -> require checkpoint visibility for the working directory -> read the changed files from the host's write record for the command -> ask visibility in batches of 256, select at most 8, check up to 4 concurrently -> append ordered diagnostics, preserving exit status and file metadata ``` Existing syntax failures remain visible after unrelated edits and identical rewrites. A baseline that cannot be read, parsed or analyzed is marked unknown; overwriting non-UTF-8 source still checks the known replacement. Source nested too deeply for the parser or checker, and a changed file that cannot be read, report diagnostics unavailable for that file rather than passing as clean. Every check reads a file as the bytes Python runs: text a write or edit stores is checked as its UTF-8 bytes, the content it replaced included, and a file from disk as its bytes, each decoded as Python decodes source, by its coding declaration or as UTF-8. Bytes that do not decode are reported as a syntax error at the line where decoding stops, so a declaration the written text cannot satisfy is reported however the file was changed. Noqa comments are read as Ruff reads them, including after other comment directives and never inside string literals: a file-level `ruff: noqa` / `flake8: noqa` counts only on a line of its own, and a marker counts only when codes follow its colon or, with no colon, whitespace, another comment or the end of the comment follows it. Unlike Ruff, which takes the first directive in a comment, every directive in it counts. The seeded tool guide gives the syntax note's own advice. With no checkpoint or no visibility verdict, exec diagnostics do not inspect changed file contents; direct file checks remain available. Pyflakes is imported when the tool face loads, not by whichever check first needs it: imported lazily, its cold import fell inside that check's one-second wait, and on a slow Windows runner the one test that paid it reported diagnostics unavailable for a plain syntax error. Private workers bound reads and checks to four active jobs per process. Saturated capacity rejects new work without queuing, and timeout/cancellation retains each slot until the actual work finishes. Name checks are capped at 256 KiB of source and the syntax check at 2 MiB, and a check skipped for size says so in one line. Findings are capped at eight per file, and exec diagnostic text at 6,000 characters plus omission notices. Runtime changes stay inside the Raven-Code code-flow plugin, except for one fix to `CheckpointService.trackable`, the visibility verdict that exec diagnostics and the host's exec write record both rely on. On macOS, git precomposes a file name it reads off the disk before matching an ignore rule, but takes the paths `trackable` sends to `git check-ignore --stdin` as written. A file stored with a decomposed (NFD) name under a `.gitignore` rule written precomposed (NFC) is therefore left out of the checkpoint, yet main vouches for it, so the write record's diff carries its text, and exec diagnostics would quote its source lines to the model. Nor is git's precomposition NFC: it composes only neighbouring characters, by Unicode 3.2's table, and reorders nothing, so a name whose combining marks are out of canonical order comes out as neither the name as given nor its NFC. `trackable` now asks about each name as given, as NFC and in git's macOS spelling, and withholds it when any spelling is ignored. Other shared file/shell/snapshot implementations are unchanged. Raven-Code has no separate dependency manifest, so Pyflakes uses the existing root dependencies. It is the only added package; uv normalized existing lockfile markers without changing other package versions. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - **Before:** on main, Raven-Code's file tools report Python syntax errors only, and its exec reports nothing about the Python files a command changed. On macOS, main's `CheckpointService.trackable` vouches for a file stored as `secret_cafe` + U+0301 + `.py` under the rule `secret_caf` + U+00E9 + `.py`: the user's own git ignores it and the checkpoint's staged tree leaves it out, yet its text reaches the exec write record's stored diff. - **After:** on macOS 26.6.2 / arm64 with Apple Git 2.54.0, the command below passes 364 tests on Python 3.12.13 and on 3.14.6. The full default suite on 3.12.13 has 32283 passed, 169 skipped, 20 failed and 3 errors over 22 test ids: 21 of them also fail on the exact base in the same environment, and the other, a remote process-tree cancel test that missed its deadline under the suite's load, passes when run alone. Tests cover actual worker overlap, retained capacity after timeout/cancellation, host-tool isolation, hidden-file quotas, visibility batching through the host's write-record wiring, baseline handling, parser-limit failures, the size bounds and their skip notes, undecodable bytes placed where decoding stops, coding declarations in written and edited text, the baseline read staying on a worker, the checker loaded with its module, the file each unavailable note names, comment directives and their marker boundary, the tool guide's agreement with the syntax note, portable commands in paths with spaces/non-ASCII characters, and the checkpoint's verdict on decomposed names and on names git spells as neither given nor NFC, held against its own staged tree. Removing any of the three spellings from the query fails the cases that pin it, as does removing the marker boundary, the parser-limit report, the unknown-baseline handling, the syntax check past the name-check bound and the decoding report, checking written text as its stored bytes, the worker around the baseline read, loading the checker with its module, or the file name in an unavailable note; restoring the guide's earlier advice fails its agreement test. git's macOS spelling is what libiconv's UTF-8-MAC conversion returns, which git precomposes with: composing neighbours by Unicode 3.2's table reproduced it for all 281,928 names built from every decomposable character's decomposition (reordered, partly composed, or with a mark added), and across 784,220 further mixed names and pairs no name was spelled outside the three asked; the same held under Python 3.13 and 3.14. The decomposed-name check asserts the macOS outcome only where git precomposes names, so it also passes with `core.precomposeunicode = false` set through `GIT_CONFIG_GLOBAL`. Across 102 noqa comment variants the reported lines match Ruff 0.15.12, except the three that hold several directives in one comment. Through the real `AgentLoop`, code-flow tool face, `CodeExecutor` and checkpoint, with the working directory under `/tmp`, under `$TMPDIR` and in a path with spaces and CJK characters, exec diagnostics report created, modified and decomposed-name files, follow `working_dir`, report an undecodable file at the line where decoding stops, a syntax error past 256 KiB and a skip past 2 MiB, and withhold `.gitignore`d files, including decomposed names and names git spells as neither given nor NFC; across repeated runs on both interpreters, a file near the 256 KiB cap is checked in 52-135 ms (117-128 ms against a baseline), the syntax of a file near 2 MiB in 318-371 ms, and 600 ignored files plus one visible file take 73-90 ms with the event loop held at most 20 ms. At this head CI passes the unit suite on Linux/Python 3.12 (32619 tests over four shards), and its two Windows steps pass the diagnostic files on native Windows/Python 3.13: 122 file/worker tests, and 27 exec diagnostic tests with 17 pre-existing executor tests deselected; the symlink fixture skips only where the host cannot create links. ```sh env -u SERPER_API_KEY -u RAVEN_MIRRORED_SERPER_API_KEY uv run --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_flow_exec.py tests/test_agents_code_flow_plugin.py tests/test_agents_code_flow_read_state.py tests/test_agents_code_launcher.py tests/test_workdir_snapshot.py tests/test_shell_command_writes.py tests/test_runtime_checkpoint.py tests/test_trajectory_upload.py::test_blocked_upload_hits_the_deadline -o addopts= -q --tb=short --maxfail=3 uv run --no-sync ruff check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_diagnostic_workers.py uv run --no-sync ty check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py uv run --no-sync lint-imports uv run --no-sync ruff format --check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py ``` Native Windows CI commands: ```sh uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py -q -n 0 --tb=short uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_flow_exec.py -k test_exec_ -q -n 0 --tb=short ``` Ruff, format and type checks passed; all 10 import contracts kept; the commit-message, large-file and source-language checks pass over the branch. The launcher tests unset inherited search API keys to keep the expected tool list deterministic. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed ## Risk Scope: Raven-Code's enabled code-flow tool face; installing Pyflakes does not activate checks in other agents. The change is reversible and never fixes/removes code or rolls back edits. Static diagnostics can need judgment for dynamic names, target Python versions, and project lint configuration; only explicit noqa directives are honored. Passing diagnostics do not replace tests. The `trackable` fix reaches every exec write record, not only Raven-Code: a file whose name, as given, as NFC or in git's macOS spelling, matches an ignore rule keeps its counts and loses its diff and removal body. Where git matches another of those spellings, as off macOS for a decomposed name under a precomposed rule, the checkpoint stores the file but it is still withheld; its counts remain. Exec diagnostics now require an active checkpoint. Noninteractive runs using the default interactive checkpoint policy must explicitly enable checkpoints to receive this feedback. Visibility is checked in batches of up to 256 files and stops once eight visible files are found; each batch holds the event loop while the host resolves its paths, about 130 ms for a full batch on native Windows. Each file-check wait is one second, and post-command diagnostics have a three-second wait budget; past it, or when they fail as a whole, they are dropped and the command's result is kept. These are caller budgets, not hard I/O deadlines: running OS reads and inherited synchronous host callbacks cannot be forcibly interrupted. Private worker capacity remains bounded. Between 256 KiB and 2 MiB only syntax is checked: near 2 MiB that took 0.3-0.4 s and about 190 MiB here, up to four checks run at once, and on slower hardware such a check can outlast the one-second wait and report diagnostics unavailable. Python's GIL means concurrency does not promise a CPU speedup. Windows CI exercises the diagnostics layer with the real native DirectExecutor. The existing CodeExecutor environment probe uses POSIX quoting and already fails on Windows at the base; this PR neither changes that probe nor claims end-to-end compatibility for it. Symlink setup is skipped only if the host cannot create the fixture. Disable `pythonLint` for syntax-only feedback, `execPythonDiagnostics` for no post-command checks, or `pythonSyntaxNote` for no diagnostics, under `plugins.config["code-flow"].tools`. The PR can also be reverted as a unit. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues N/A --------- Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn> Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
101 lines
3.4 KiB
Python
101 lines
3.4 KiB
Python
"""Exercise actual Raven-Code workers, including work surviving caller cancellation."""
|
|
|
|
import asyncio
|
|
import contextvars
|
|
import sys
|
|
import threading
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
|
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "agents/raven-code/plugins/code-flow"))
|
|
from code_flow.tools.diagnostic_workers import DiagnosticWorkers, WorkersBusyError # noqa: E402
|
|
|
|
|
|
async def _until(predicate):
|
|
async def poll():
|
|
while not predicate():
|
|
await asyncio.sleep(0.001)
|
|
|
|
await asyncio.wait_for(poll(), 2)
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
@pytest.mark.parametrize("stop", ["timeout", "cancel"])
|
|
async def test_abandoned_work_retains_capacity_until_it_really_finishes(stop):
|
|
workers = DiagnosticWorkers(limit=1)
|
|
entered, release, finished = threading.Event(), threading.Event(), threading.Event()
|
|
|
|
def stalled():
|
|
entered.set()
|
|
try:
|
|
release.wait(3)
|
|
finally:
|
|
finished.set()
|
|
|
|
task = asyncio.create_task(workers.run(stalled, timeout=0.1 if stop == "timeout" else 5))
|
|
try:
|
|
await _until(entered.is_set)
|
|
if stop == "cancel":
|
|
task.cancel()
|
|
with pytest.raises(TimeoutError if stop == "timeout" else asyncio.CancelledError):
|
|
await task
|
|
calls = []
|
|
for _ in range(12):
|
|
with pytest.raises(WorkersBusyError):
|
|
await workers.run(lambda: calls.append("unexpected queued work"), timeout=1)
|
|
assert not calls and not finished.is_set()
|
|
finally:
|
|
release.set()
|
|
await _until(finished.is_set)
|
|
|
|
async def recover():
|
|
while True:
|
|
try:
|
|
return await workers.run(lambda: "recovered", timeout=1)
|
|
except WorkersBusyError:
|
|
await asyncio.sleep(0.001)
|
|
|
|
assert await asyncio.wait_for(recover(), 2) == "recovered"
|
|
assert not calls
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_workers_overlap_without_using_the_hosts_executor_and_keep_context(monkeypatch):
|
|
workers = DiagnosticWorkers(limit=4)
|
|
barrier = threading.Barrier(4, timeout=2)
|
|
tag = contextvars.ContextVar("diagnostic_test_tag", default="missing")
|
|
token = tag.set("caller")
|
|
loop_thread = threading.get_ident()
|
|
|
|
def forbidden(*args, **kwargs):
|
|
raise AssertionError("diagnostics used the host's executor")
|
|
|
|
def check():
|
|
barrier.wait()
|
|
return threading.get_ident(), threading.current_thread().daemon, tag.get()
|
|
|
|
monkeypatch.setattr(asyncio.get_running_loop(), "run_in_executor", forbidden)
|
|
try:
|
|
results = await asyncio.gather(*(workers.run(check, timeout=3) for _ in range(4)))
|
|
finally:
|
|
tag.reset(token)
|
|
assert len({ident for ident, _, _ in results}) == 4
|
|
assert all(ident != loop_thread and daemon and value == "caller" for ident, daemon, value in results)
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_worker_exception_and_thread_start_failure_release_capacity(monkeypatch):
|
|
workers = DiagnosticWorkers(limit=1)
|
|
|
|
def broken(*args):
|
|
raise RuntimeError("worker failed")
|
|
|
|
with pytest.raises(RuntimeError, match="worker failed"):
|
|
await workers.run(broken, timeout=1)
|
|
with monkeypatch.context() as patch:
|
|
patch.setattr(threading.Thread, "start", broken)
|
|
with pytest.raises(RuntimeError, match="worker failed"):
|
|
await workers.run(lambda: 0, timeout=1)
|
|
assert await workers.run(lambda: 1, timeout=1) == 1
|