Raven-1/tests/test_agents_code_tools_plugin.py
Tong Li 0a1a711123
feat(*): add bounded Python diagnostics after code edits (#967)
## Summary

Raven-Code reports Python syntax errors after file edits but misses
unresolved names and equivalent feedback after shell edits. Add advisory
Pyflakes F821/F822/F823 checks to the existing tool result; optional
F401 notices remain off by default. No additional model call or code
execution is involved.

```text
write_file / edit_file
  -> inspect final source, compare with a baseline that can be read and analyzed
  -> retain syntax warnings; suppress existing name findings
exec
  -> require checkpoint visibility for the working directory
  -> read the changed files from the host's write record for the command
  -> ask visibility in batches of 256, select at most 8, check up to 4 concurrently
  -> append ordered diagnostics, preserving exit status and file metadata
```

Existing syntax failures remain visible after unrelated edits and
identical rewrites. A baseline that cannot be read, parsed or analyzed
is marked unknown; overwriting non-UTF-8 source still checks the known
replacement. Source nested too deeply for the parser or checker, and a
changed file that cannot be read, report diagnostics unavailable for
that file rather than passing as clean. Every check reads a file as the
bytes Python runs: text a write or edit stores is checked as its UTF-8
bytes, the content it replaced included, and a file from disk as its
bytes, each decoded as Python decodes source, by its coding declaration
or as UTF-8. Bytes that do not decode are reported as a syntax error at
the line where decoding stops, so a declaration the written text cannot
satisfy is reported however the file was changed. Noqa comments are read
as Ruff reads them, including after other comment directives and never
inside string literals: a file-level `ruff: noqa` / `flake8: noqa`
counts only on a line of its own, and a marker counts only when codes
follow its colon or, with no colon, whitespace, another comment or the
end of the comment follows it. Unlike Ruff, which takes the first
directive in a comment, every directive in it counts. The seeded tool
guide gives the syntax note's own advice. With no checkpoint or no
visibility verdict, exec diagnostics do not inspect changed file
contents; direct file checks remain available.

Pyflakes is imported when the tool face loads, not by whichever check
first needs it: imported lazily, its cold import fell inside that
check's one-second wait, and on a slow Windows runner the one test that
paid it reported diagnostics unavailable for a plain syntax error.
Private workers bound reads and checks to four active jobs per process.
Saturated capacity rejects new work without queuing, and
timeout/cancellation retains each slot until the actual work finishes.
Name checks are capped at 256 KiB of source and the syntax check at 2
MiB, and a check skipped for size says so in one line. Findings are
capped at eight per file, and exec diagnostic text at 6,000 characters
plus omission notices.

Runtime changes stay inside the Raven-Code code-flow plugin, except for
one fix to `CheckpointService.trackable`, the visibility verdict that
exec diagnostics and the host's exec write record both rely on. On
macOS, git precomposes a file name it reads off the disk before matching
an ignore rule, but takes the paths `trackable` sends to `git
check-ignore --stdin` as written. A file stored with a decomposed (NFD)
name under a `.gitignore` rule written precomposed (NFC) is therefore
left out of the checkpoint, yet main vouches for it, so the write
record's diff carries its text, and exec diagnostics would quote its
source lines to the model. Nor is git's precomposition NFC: it composes
only neighbouring characters, by Unicode 3.2's table, and reorders
nothing, so a name whose combining marks are out of canonical order
comes out as neither the name as given nor its NFC. `trackable` now asks
about each name as given, as NFC and in git's macOS spelling, and
withholds it when any spelling is ignored. Other shared
file/shell/snapshot implementations are unchanged.

Raven-Code has no separate dependency manifest, so Pyflakes uses the
existing root dependencies. It is the only added package; uv normalized
existing lockfile markers without changing other package versions.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- **Before:** on main, Raven-Code's file tools report Python syntax
errors only, and its exec reports nothing about the Python files a
command changed. On macOS, main's `CheckpointService.trackable` vouches
for a file stored as `secret_cafe` + U+0301 + `.py` under the rule
`secret_caf` + U+00E9 + `.py`: the user's own git ignores it and the
checkpoint's staged tree leaves it out, yet its text reaches the exec
write record's stored diff.
- **After:** on macOS 26.6.2 / arm64 with Apple Git 2.54.0, the command
below passes 364 tests on Python 3.12.13 and on 3.14.6. The full default
suite on 3.12.13 has 32283 passed, 169 skipped, 20 failed and 3 errors
over 22 test ids: 21 of them also fail on the exact base in the same
environment, and the other, a remote process-tree cancel test that
missed its deadline under the suite's load, passes when run alone. Tests
cover actual worker overlap, retained capacity after
timeout/cancellation, host-tool isolation, hidden-file quotas,
visibility batching through the host's write-record wiring, baseline
handling, parser-limit failures, the size bounds and their skip notes,
undecodable bytes placed where decoding stops, coding declarations in
written and edited text, the baseline read staying on a worker, the
checker loaded with its module, the file each unavailable note names,
comment directives and their marker boundary, the tool guide's agreement
with the syntax note, portable commands in paths with spaces/non-ASCII
characters, and the checkpoint's verdict on decomposed names and on
names git spells as neither given nor NFC, held against its own staged
tree. Removing any of the three spellings from the query fails the cases
that pin it, as does removing the marker boundary, the parser-limit
report, the unknown-baseline handling, the syntax check past the
name-check bound and the decoding report, checking written text as its
stored bytes, the worker around the baseline read, loading the checker
with its module, or the file name in an unavailable note; restoring the
guide's earlier advice fails its agreement test. git's macOS spelling is
what libiconv's UTF-8-MAC conversion returns, which git precomposes
with: composing neighbours by Unicode 3.2's table reproduced it for all
281,928 names built from every decomposable character's decomposition
(reordered, partly composed, or with a mark added), and across 784,220
further mixed names and pairs no name was spelled outside the three
asked; the same held under Python 3.13 and 3.14. The decomposed-name
check asserts the macOS outcome only where git precomposes names, so it
also passes with `core.precomposeunicode = false` set through
`GIT_CONFIG_GLOBAL`. Across 102 noqa comment variants the reported lines
match Ruff 0.15.12, except the three that hold several directives in one
comment. Through the real `AgentLoop`, code-flow tool face,
`CodeExecutor` and checkpoint, with the working directory under `/tmp`,
under `$TMPDIR` and in a path with spaces and CJK characters, exec
diagnostics report created, modified and decomposed-name files, follow
`working_dir`, report an undecodable file at the line where decoding
stops, a syntax error past 256 KiB and a skip past 2 MiB, and withhold
`.gitignore`d files, including decomposed names and names git spells as
neither given nor NFC; across repeated runs on both interpreters, a file
near the 256 KiB cap is checked in 52-135 ms (117-128 ms against a
baseline), the syntax of a file near 2 MiB in 318-371 ms, and 600
ignored files plus one visible file take 73-90 ms with the event loop
held at most 20 ms. At this head CI passes the unit suite on
Linux/Python 3.12 (32619 tests over four shards), and its two Windows
steps pass the diagnostic files on native Windows/Python 3.13: 122
file/worker tests, and 27 exec diagnostic tests with 17 pre-existing
executor tests deselected; the symlink fixture skips only where the host
cannot create links.

```sh
env -u SERPER_API_KEY -u RAVEN_MIRRORED_SERPER_API_KEY uv run --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_flow_exec.py tests/test_agents_code_flow_plugin.py tests/test_agents_code_flow_read_state.py tests/test_agents_code_launcher.py tests/test_workdir_snapshot.py tests/test_shell_command_writes.py tests/test_runtime_checkpoint.py tests/test_trajectory_upload.py::test_blocked_upload_hits_the_deadline -o addopts= -q --tb=short --maxfail=3
uv run --no-sync ruff check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_diagnostic_workers.py
uv run --no-sync ty check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py
uv run --no-sync lint-imports
uv run --no-sync ruff format --check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py
```

Native Windows CI commands:

```sh
uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py -q -n 0 --tb=short
uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_flow_exec.py -k test_exec_ -q -n 0 --tb=short
```

Ruff, format and type checks passed; all 10 import contracts kept; the
commit-message, large-file and source-language checks pass over the
branch. The launcher tests unset inherited search API keys to keep the
expected tool list deterministic.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Scope: Raven-Code's enabled code-flow tool face; installing Pyflakes
does not activate checks in other agents. The change is reversible and
never fixes/removes code or rolls back edits. Static diagnostics can
need judgment for dynamic names, target Python versions, and project
lint configuration; only explicit noqa directives are honored. Passing
diagnostics do not replace tests.

The `trackable` fix reaches every exec write record, not only
Raven-Code: a file whose name, as given, as NFC or in git's macOS
spelling, matches an ignore rule keeps its counts and loses its diff and
removal body. Where git matches another of those spellings, as off macOS
for a decomposed name under a precomposed rule, the checkpoint stores
the file but it is still withheld; its counts remain.

Exec diagnostics now require an active checkpoint. Noninteractive runs
using the default interactive checkpoint policy must explicitly enable
checkpoints to receive this feedback. Visibility is checked in batches
of up to 256 files and stops once eight visible files are found; each
batch holds the event loop while the host resolves its paths, about 130
ms for a full batch on native Windows. Each file-check wait is one
second, and post-command diagnostics have a three-second wait budget;
past it, or when they fail as a whole, they are dropped and the
command's result is kept. These are caller budgets, not hard I/O
deadlines: running OS reads and inherited synchronous host callbacks
cannot be forcibly interrupted. Private worker capacity remains bounded.
Between 256 KiB and 2 MiB only syntax is checked: near 2 MiB that took
0.3-0.4 s and about 190 MiB here, up to four checks run at once, and on
slower hardware such a check can outlast the one-second wait and report
diagnostics unavailable. Python's GIL means concurrency does not promise
a CPU speedup.

Windows CI exercises the diagnostics layer with the real native
DirectExecutor. The existing CodeExecutor environment probe uses POSIX
quoting and already fails on Windows at the base; this PR neither
changes that probe nor claims end-to-end compatibility for it. Symlink
setup is skipped only if the host cannot create the fixture.

Disable `pythonLint` for syntax-only feedback, `execPythonDiagnostics`
for no post-command checks, or `pythonSyntaxNote` for no diagnostics,
under `plugins.config["code-flow"].tools`. The PR can also be reverted
as a unit.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn>
Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-18 20:21:04 +08:00

1349 lines
60 KiB
Python

"""code-flow's tool face (code_flow/tools/): the fork's tools as same-name replacements.
Pins the manifest rows, the D6 admission from the slice, the fork's schema
spelling with the host's names accepted as aliases, the two pieces of conduct
the file tools carry (read-before-edit, the Python syntax verdict), the
``occurrence`` edit, ``glob`` and ``todo``, and that a plugin tool
registered after the built-in replaces it in the registry the loop reads.
"""
import asyncio
import itertools
import os
import subprocess
import sys
import threading
from pathlib import Path
from types import SimpleNamespace
import pytest
from raven.agent import workdir
from raven.agent.tools.filesystem import ReadFileTool as TrunkReadFileTool
from raven.agent.tools.registry import ToolRegistry
from raven.contracts.tool import ToolResult
from raven.plugins import DiscoveredPlugin, ManifestOrigin, PluginManifest, PluginRegistry
from raven.plugins.context import PluginContext, ServiceLocator
REPO = Path(__file__).resolve().parent.parent
PLUGIN_DIR = REPO / "agents" / "raven-code" / "plugins" / "code-flow"
sys.path.insert(0, str(PLUGIN_DIR))
from code_flow.tools import plugin as factories # noqa: E402
from code_flow.tools.filesystem import EditFileTool, ReadFileTool, ReadLedger, WriteFileTool # noqa: E402
from code_flow.tools.read_state import ReadSessions, owner_for # noqa: E402
from code_flow.tools.search import GlobTool, expand_braces, is_notice # noqa: E402
MANIFEST_PATH = PLUGIN_DIR / "raven-plugin.toml"
FACE = ("read_file", "write_file", "edit_file", "list_dir", "glob", "todo", "exec")
def _ctx(tmp_path: Path, tools: dict) -> PluginContext:
"""A plugin context carrying the flow slice whose ``tools`` section is the
product's own tool face."""
return PluginContext(
config={"enabled": True, "tools": tools},
services=ServiceLocator(workspace=tmp_path, user_id="u", agent_id="a"),
)
def trunk_read_file(workspace: Path):
from raven.agent.tools.filesystem import ReadFileTool as TrunkRead
return TrunkRead(workspace=workspace)
def registry_execute(reg: ToolRegistry, name: str, params: dict):
return reg.execute(name, params)
def _run(coro):
return asyncio.get_event_loop_policy().new_event_loop().run_until_complete(coro)
def _text(result) -> str:
return result.model_text if isinstance(result, ToolResult) else str(result)
@pytest.fixture(autouse=True)
def _fresh_ledger(tmp_path):
reads = ReadSessions(owner_for(_ctx(tmp_path, {"enabled": True})))
reads.bind("cli:unit")
yield
reads.unbind()
@pytest.fixture()
def registry(tmp_path):
ctx = _ctx(tmp_path, {"enabled": True})
reg = ToolRegistry()
for make in (
factories.make_read_file,
factories.make_write_file,
factories.make_edit_file,
factories.make_list_dir,
factories.make_glob,
factories.make_todo,
factories.make_exec,
):
reg.register(make(ctx))
return reg
# --- manifest and admission ----------------------------------------------------
def test_the_manifest_contributes_the_seven_tools_beside_the_flows_own_seats():
"""One plugin per product, the way every sibling ships: the tool face is a
subpackage of code-flow, not a plugin of its own."""
mf = PluginManifest.from_toml_path(MANIFEST_PATH)
assert mf.id == "code-flow"
assert [t.name for t in mf.contributes.tools] == list(FACE)
assert [h.name for h in mf.contributes.hooks] == ["code_flow"]
assert mf.contributes.tool_gates == []
assert [f.factory for f in mf.contributes.tools] == [f"code_flow.tools.plugin:make_{name}" for name in FACE]
def test_the_registry_builds_every_tool_from_the_real_factory_strings(tmp_path):
reg = PluginRegistry()
mf = PluginManifest.from_toml_path(MANIFEST_PATH)
reg.activate([DiscoveredPlugin(manifest=mf, source=ManifestOrigin.USER, location=MANIFEST_PATH)])
assert reg.tool_names() == sorted(FACE)
services = ServiceLocator(workspace=tmp_path, user_id="u", agent_id="a")
slice_ = {"enabled": True, "tools": {"enabled": True}}
for name in FACE:
tool = reg.build_tool(name, config=slice_, services=services)
assert tool is not None and tool.name == name
@pytest.mark.parametrize(
"config",
[
{},
{"enabled": True},
{"enabled": True, "tools": {"enabled": False}},
{"tools": {"pythonLint": True, "execPythonDiagnostics": True}},
{"tools": {"enabled": False, "pythonLint": True, "execPythonDiagnostics": True}},
],
)
def test_a_slice_that_does_not_ask_for_the_face_contributes_nothing(tmp_path, config):
"""The flow and the face have one switch each: a product can keep the flow
and hand the tools back to the host."""
ctx = PluginContext(config=config, services=ServiceLocator(workspace=tmp_path, user_id="u", agent_id="a"))
for make in (
factories.make_read_file,
factories.make_write_file,
factories.make_edit_file,
factories.make_list_dir,
factories.make_glob,
factories.make_todo,
factories.make_exec,
):
assert make(ctx) is None
@pytest.mark.asyncio
async def test_loading_coding_tools_does_not_add_diagnostics_to_host_tools(tmp_path, monkeypatch):
from code_flow.tools.diagnostics import PythonDiagnostics
from raven.agent.tools.filesystem import WriteFileTool as TrunkWriteFileTool
from raven.agent.tools.shell import ExecTool as TrunkExecTool
calls = []
async def check(*args, **kwargs):
calls.append(args)
return "unexpected coding diagnostics"
monkeypatch.setattr(PythonDiagnostics, "check", check)
writer = TrunkWriteFileTool(workspace=tmp_path)
out = await writer.execute(path=str(tmp_path / "host.py"), content="value = missing_binding\n")
assert "Successfully wrote" in str(out)
shell = TrunkExecTool(working_dir=str(tmp_path))
out = await shell.execute(command="echo other = another_missing > shell.py")
assert "Exit code: 0" in str(out)
assert calls == []
assert "another_missing" in (tmp_path / "shell.py").read_text()
# --- the fork's schema, the host's names as aliases ----------------------------
def test_the_schemas_carry_the_forks_spelling(registry):
read = registry.get("read_file").parameters
assert read["required"] == ["file_path"] and "path" not in read["properties"]
edit = registry.get("edit_file").parameters
assert edit["required"] == ["file_path", "old_string", "new_string"]
assert "occurrence" in edit["properties"] and "old_text" not in edit["properties"]
write = registry.get("write_file").parameters
assert write["required"] == ["file_path", "content"]
assert registry.get("glob").name == "glob"
assert registry.get("find") is None
def test_the_hosts_spelling_still_runs_through_the_aliases(registry, tmp_path):
target = tmp_path / "a.txt"
target.write_text("alpha beta\n")
assert "1| alpha beta" in _text(_run(registry.execute("read_file", {"path": str(target)})))
result = _run(registry.execute("edit_file", {"path": str(target), "old_text": "beta", "new_text": "gamma"}))
assert "Successfully edited" in _text(result)
assert target.read_text() == "alpha gamma\n"
def test_the_forks_spelling_wins_when_a_call_carries_both(tmp_path):
tool = EditFileTool(workspace=tmp_path)
cast = tool.cast_params({"path": "legacy.py", "file_path": "fork.py", "old_text": "a", "old_string": "b"})
assert cast == {"file_path": "fork.py", "old_string": "b"}
# --- read before edit ---------------------------------------------------------------
def test_editing_a_file_nobody_read_is_refused_until_it_is_read(registry, tmp_path):
target = tmp_path / "b.txt"
target.write_text("one\n")
refused = _text(
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "one", "new_string": "two"}))
)
assert refused.startswith("Error") and "have not read" in refused
assert target.read_text() == "one\n"
_run(registry.execute("read_file", {"file_path": str(target)}))
accepted = _run(registry.execute("edit_file", {"file_path": str(target), "old_string": "one", "new_string": "two"}))
assert "Successfully edited" in _text(accepted)
assert target.read_text() == "two\n"
def test_a_write_counts_as_having_read_what_now_stands(registry, tmp_path):
target = tmp_path / "c.txt"
_run(registry.execute("write_file", {"file_path": str(target), "content": "x = 1\n"}))
result = _run(registry.execute("edit_file", {"file_path": str(target), "old_string": "1", "new_string": "2"}))
assert "Successfully edited" in _text(result)
def test_the_read_rule_is_a_switch(tmp_path):
ctx = _ctx(tmp_path, {"enabled": True, "requireReadBeforeEdit": False})
tool = factories.make_edit_file(ctx)
target = tmp_path / "d.txt"
target.write_text("k\n")
assert "Successfully edited" in _text(_run(tool.execute(file_path=str(target), old_string="k", new_string="v")))
# --- the syntax verdict ---------------------------------------------------------------
def test_a_write_that_breaks_python_says_so_in_the_same_result(registry, tmp_path):
target = tmp_path / "e.py"
result = _text(_run(registry.execute("write_file", {"file_path": str(target), "content": "def f(:\n"})))
assert "Successfully wrote" in result
assert "syntax error" in result and "line 1" in result
def test_an_edit_that_breaks_python_says_so_and_a_clean_one_stays_quiet(registry, tmp_path):
target = tmp_path / "f.py"
target.write_text("x = 1\n")
_run(registry.execute("read_file", {"file_path": str(target)}))
broken = _text(
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "1", "new_string": "("}))
)
assert "syntax error" in broken
fixed = _text(_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "(", "new_string": "2"})))
assert "Successfully edited" in fixed and "syntax error" not in fixed
def test_the_verdict_is_python_only_and_a_switch(tmp_path):
ctx = _ctx(tmp_path, {"enabled": True})
write = factories.make_write_file(ctx)
assert "syntax" not in _text(_run(write.execute(file_path=str(tmp_path / "g.txt"), content="def f(:\n")))
quiet = factories.make_write_file(_ctx(tmp_path, {"enabled": True, "pythonSyntaxNote": False}))
assert "syntax" not in _text(_run(quiet.execute(file_path=str(tmp_path / "h.py"), content="def f(:\n")))
def test_undefined_name_feedback_identifies_the_binding_and_keeps_the_edit(registry, tmp_path):
target = tmp_path / "undefined.py"
source = "def double():\n return missing_value * 2\n"
result = _text(_run(registry.execute("write_file", {"file_path": str(target), "content": source})))
assert "F821" in result and "missing_value" in result and "line 2" in result
assert "spelling, scope, import, or assignment order" in result
assert target.read_text() == source
def test_existing_name_errors_do_not_reappear_when_unrelated_lines_move(registry, tmp_path):
target = tmp_path / "existing.py"
target.write_text("old_result = preexisting_missing\nvalue = 1\n")
_run(registry.execute("read_file", {"file_path": str(target)}))
result = _text(
_run(
registry.execute(
"edit_file",
{
"file_path": str(target),
"old_string": "old_result =",
"new_string": "\nold_result =",
},
)
)
)
assert "F821" not in result
result = _text(
_run(
registry.execute(
"edit_file",
{
"file_path": str(target),
"old_string": "value = 1",
"new_string": "value = newly_missing",
},
)
)
)
assert "F821" in result and "newly_missing" in result
assert "preexisting_missing" not in result
@pytest.mark.parametrize("tool_name", ["edit_file", "write_file"])
def test_existing_syntax_error_is_still_reported_after_an_unrelated_edit(registry, tmp_path, tool_name):
target = tmp_path / "still_broken.py"
source = "def f(:\n return 1\nvalue = 2\n"
_run(registry.execute("write_file", {"file_path": str(target), "content": source}))
params = {"file_path": str(target)}
if tool_name == "edit_file":
params.update(old_string="value = 2", new_string="value = 3")
else:
params["content"] = source.replace("value = 2", "value = 3")
result = _text(_run(registry.execute(tool_name, params)))
assert "Python syntax error" in result
assert "new findings compared" not in result
assert target.read_text() == source.replace("value = 2", "value = 3")
def test_fixing_syntax_does_not_label_old_name_errors_as_new(registry, tmp_path):
target = tmp_path / "syntax_fixed.py"
source = "value = old_missing\ndef f(:\n return 1\n"
_run(registry.execute("write_file", {"file_path": str(target), "content": source}))
result = _text(
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "f(:", "new_string": "f():"}))
)
assert "F821" in result and "old_missing" in result
assert "baseline is unknown" in result and "new findings compared" not in result
@pytest.mark.parametrize("source, expected", [("def f(:\n", "Python syntax error"), ("x = missing_name\n", "F821")])
def test_overwriting_non_utf8_source_still_checks_the_new_content(registry, tmp_path, source, expected):
target = tmp_path / "legacy.py"
target.write_bytes('# coding: gbk\nname = "\u4e2d"\n'.encode("gbk"))
result = _run(registry.execute("write_file", {"file_path": str(target), "content": source}))
assert result.file_change is None
assert expected in _text(result) and "baseline is unknown" in _text(result)
assert target.read_text(encoding="utf-8") == source
def test_import_notices_are_optional_and_explain_side_effects(tmp_path):
source = "import os\n"
quiet = factories.make_write_file(_ctx(tmp_path, {"enabled": True}))
result = _text(_run(quiet.execute(file_path=str(tmp_path / "quiet_import.py"), content=source)))
assert "F401" not in result
enabled = factories.make_write_file(_ctx(tmp_path, {"enabled": True, "pythonUnusedImports": True}))
target = tmp_path / "notice_import.py"
result = _text(_run(enabled.execute(file_path=str(target), content=source)))
assert "NOTICE" in result and "F401" in result and "side effects" in result
assert target.read_text() == source
def test_name_checks_handle_nested_scopes_and_public_exports(registry, tmp_path):
source = (
"from os import path\n__all__ = ['path']\n"
"def outer(value):\n def inner():\n return value + len([])\n return inner\n"
)
result = _text(
_run(
registry.execute(
"write_file",
{
"file_path": str(tmp_path / "scopes.py"),
"content": source,
},
)
)
)
assert "Python diagnostics" not in result
def test_name_checks_can_be_disabled_without_disabling_syntax_feedback(tmp_path):
tool = factories.make_write_file(_ctx(tmp_path, {"enabled": True, "pythonLint": False}))
target = tmp_path / "syntax_only.py"
assert "F821" not in _text(_run(tool.execute(file_path=str(target), content="result = missing\n")))
assert "syntax error" in _text(_run(tool.execute(file_path=str(target), content="def f(:\n")))
def test_append_checks_the_complete_file_and_suppresses_preexisting_errors(registry, tmp_path):
target = tmp_path / "append.py"
target.write_text("old = existing_missing\n")
result = _text(
_run(
registry.execute(
"write_file",
{
"file_path": str(target),
"content": "new = added_missing\n",
"mode": "append",
},
)
)
)
assert "added_missing" in result and "existing_missing" not in result
@pytest.mark.parametrize(
("source", "code"),
[
("__all__ = ['missing_export']\n", "F822"),
("value = 1\ndef f():\n print(value)\n value = 2\n", "F823"),
],
)
def test_name_feedback_covers_exports_and_use_before_local_assignment(registry, tmp_path, source, code):
target = tmp_path / "binding.py"
out = _text(_run(registry.execute("write_file", {"file_path": str(target), "content": source})))
assert code in out
assert target.read_text() == source
@pytest.mark.parametrize(
("source", "warns"),
[
("value = injected_name # noqa\n", False),
("value = injected_name # noqa: F821, F401\n", False),
("value = injected_name # noqa: F401\n", True),
("value = injected_name # type: ignore[name-defined] # noqa: F821\n", False),
("value = injected_name # type: ignore[name-defined] # noqa: F401\n", True),
("value = injected_name # noqa: F401 # noqa: F821\n", False),
("value = injected_name # noqa: F401 # noqa\n", False),
("# ruff: noqa: F821\nvalue = injected_name\n", False),
("# flake8: noqa\nvalue = injected_name\n", False),
("value = injected_name # ruff: noqa\nother = second_name\n", True),
("value = injected_name # ruff: noqa\n", True),
("value = 1 # TODO # flake8: noqa\nother = injected_name\n", True),
("label = '# noqa'; value = injected_name\n", True),
("label = '# ruff: noqa'\nvalue = injected_name\n", True),
("value = injected_name # see https://example.com/page#noqa-section\n", True),
('# Use "# ruff: noqa" only for generated files\nvalue = injected_name\n', True),
("value = injected_name # noqa: needed for registration\n", True),
("value = injected_name # noqa#reason\n", False),
("# ruff: noqa: generated file\nvalue = injected_name\n", True),
("# ruff : noqa\nvalue = injected_name\n", False),
("# Ruff: noqa\nvalue = injected_name\n", True),
],
)
def test_explicit_noqa_comments_suppress_only_the_requested_checks(registry, tmp_path, source, warns):
target = tmp_path / "ignored.py"
out = _text(_run(registry.execute("write_file", {"file_path": str(target), "content": source})))
assert ("WARNING" in out) is warns
def test_syntax_errors_are_not_hidden_by_noqa(registry, tmp_path):
source = "# ruff: noqa\ndef f(:\n"
out = _text(_run(registry.execute("write_file", {"file_path": str(tmp_path / "syntax.py"), "content": source})))
assert "Python syntax error" in out and "project's Python version" in out
def test_the_tool_guide_gives_the_syntax_note_s_advice():
"""The note and the seeded tool guide reach the model together, so the
guide must not tell it to fix outright what the note asks it to check
against the project's Python version first."""
from code_flow.tools.diagnostics import PythonDiagnostics
note = PythonDiagnostics().note(Path("broken.py"), "def f(:\n", before=None)
guide = " ".join((PLUGIN_DIR / "prompts" / "TOOLS_CODE.md").read_text(encoding="utf-8").split())
advice = "check the project's Python version before changing newer syntax"
assert advice in note and advice in guide
assert "fix it before moving on" not in guide
def test_the_name_checker_is_loaded_with_the_module_not_by_the_first_check():
"""A cold import of the checker inside a check's wait can outlast the wait
on a cold runner, which reported diagnostics unavailable for a file with a
plain syntax error. It is paid when the module loads instead. A fresh
interpreter, so no earlier test's check has already imported it."""
probe = (
"import sys; sys.path.insert(0, sys.argv[1]); import code_flow.tools.diagnostics; "
"print('pyflakes.checker' in sys.modules)"
)
done = subprocess.run([sys.executable, "-I", "-c", probe, str(PLUGIN_DIR)], capture_output=True, text=True)
assert done.returncode == 0, done.stderr
assert done.stdout.strip() == "True"
def test_checker_failure_does_not_turn_a_successful_write_into_a_failure(registry, tmp_path, monkeypatch):
from pyflakes import checker
def broken(*args, **kwargs):
raise RuntimeError("checker broke")
monkeypatch.setattr(checker, "Checker", broken)
target = tmp_path / "kept.py"
result = _run(registry.execute("write_file", {"file_path": str(target), "content": "value = 1\n"}))
assert "diagnostics unavailable (RuntimeError)" in _text(result)
assert result.file_change.after == target.read_text() == "value = 1\n"
assert "Successfully wrote" in _text(result)
@pytest.mark.parametrize("error", [RecursionError, MemoryError])
def test_source_the_parser_cannot_finish_reports_diagnostics_unavailable(registry, tmp_path, monkeypatch, error):
"""Python 3.12 raises RecursionError building the syntax tree of deeply
nested source (``compile`` too); that is no verdict, not a clean file."""
from code_flow.tools import diagnostics
def too_deep(*args, **kwargs):
raise error("nested too deeply")
monkeypatch.setattr(diagnostics, "ast", SimpleNamespace(parse=too_deep))
target = tmp_path / "deep.py"
result = _run(registry.execute("write_file", {"file_path": str(target), "content": "value = 1\n"}))
assert f"diagnostics unavailable ({error.__name__})" in _text(result)
assert target.read_text() == "value = 1\n"
@pytest.mark.parametrize("error", [RecursionError, MemoryError])
def test_replaced_source_the_parser_cannot_finish_is_an_unknown_baseline(registry, tmp_path, monkeypatch, error):
from code_flow.tools import diagnostics
parse = diagnostics.ast.parse
def deep_before(source, *args, **kwargs):
if "nested_before" in source:
raise error("nested too deeply")
return parse(source, *args, **kwargs)
target = tmp_path / "replaced.py"
target.write_text("value = nested_before\n")
monkeypatch.setattr(diagnostics, "ast", SimpleNamespace(parse=deep_before))
args = {"file_path": str(target), "content": "value = newly_missing\n"}
result = _text(_run(registry.execute("write_file", args)))
assert "newly_missing" in result and "baseline is unknown" in result
assert "new findings compared" not in result
def test_identical_rewrite_does_not_repeat_diagnostics(registry, tmp_path):
target = tmp_path / "same.py"
args = {"file_path": str(target), "content": "value = missing\n"}
assert "F821" in _text(_run(registry.execute("write_file", args)))
assert "Python diagnostics" not in _text(_run(registry.execute("write_file", args)))
def test_identical_rewrite_still_reports_a_syntax_error(registry, tmp_path):
target = tmp_path / "same_broken.py"
args = {"file_path": str(target), "content": "def f(:\n return 1\n"}
assert "Python syntax error" in _text(_run(registry.execute("write_file", args)))
result = _text(_run(registry.execute("write_file", args)))
assert result.startswith("File unchanged:") and "Python syntax error" in result
def test_diagnostics_bound_source_and_findings_without_executing_code(tmp_path):
from code_flow.tools.diagnostics import MAX_FINDINGS, MAX_SOURCE_BYTES, MAX_SYNTAX_BYTES, PythonDiagnostics
target = tmp_path / "source.py"
sentinel = tmp_path / "must_not_exist"
text = f"open({str(sentinel)!r}, 'w').close()\n"
diagnostics = PythonDiagnostics()
assert diagnostics.note(target, text, before=None) == ""
assert not sentinel.exists()
assert diagnostics.note(target, "#" * (MAX_SOURCE_BYTES + 1), before=None) == (
f"\nPython name checks skipped for {target}: the source is over 256 KiB, so only its syntax was checked."
)
assert diagnostics.note(target, "#" * (MAX_SYNTAX_BYTES + 1), before=None) == (
f"\nPython diagnostics skipped for {target}: the source is over 2 MiB. Run targeted project checks."
)
assert PythonDiagnostics(lint=False).note(target, "#" * (MAX_SOURCE_BYTES + 1), before=None) == ""
note = diagnostics.note(target, "\n".join(f"value_{i} = missing_{i}" for i in range(20)), before=None)
assert note.count("WARNING:") == MAX_FINDINGS
assert "12 more findings omitted" in note
@pytest.mark.parametrize("tool", ["write_file", "edit_file"])
def test_a_syntax_error_past_the_name_check_bound_is_still_reported(registry, tmp_path, tool):
"""Name checks stop at 256 KiB and the syntax check runs on past it, so an
edit that breaks one of a project's large modules still hears about it."""
from code_flow.tools.diagnostics import MAX_SOURCE_BYTES
target = tmp_path / "large.py"
rows = MAX_SOURCE_BYTES // len("value = 1\n") + 1
body = "value = 1\n" * rows
if tool == "write_file":
args = {"file_path": str(target), "content": body + "def broken(:\n"}
else:
clean = {"file_path": str(target), "content": body + "def kept():\n pass\n"}
assert "name checks skipped" in _text(_run(registry.execute("write_file", clean)))
args = {"file_path": str(target), "old_string": "def kept():", "new_string": "def broken(:"}
result = _text(_run(registry.execute(tool, args)))
assert f"WARNING: line {rows + 1}, column 12: Python syntax error" in result
assert "name checks skipped" not in result
def test_an_append_python_cannot_decode_is_reported_where_decoding_stops(registry, tmp_path):
"""UTF-8 appended to a file that declares GBK: Python refuses the file, so
the append hears that, at the line Python cannot decode."""
target = tmp_path / "legacy.py"
target.write_bytes('# coding: gbk\nname = "\u4e2d"\n'.encode("gbk"))
args = {"file_path": str(target), "content": 'label = "\u4e2d"\ndef broken(:\n', "mode": "append"}
result = _text(_run(registry.execute("write_file", args)))
assert "line 3, column 12: Python syntax error: 'gbk' codec can't decode byte 0xad" in result
assert "new findings compared with the content before this call" in result
@pytest.mark.parametrize(
("raw", "place", "message"),
[
# A declared encoding, where Python's parser reports line 0.
(
"# -*- coding: gbk -*-\nx = 1\nname = '\u4e2d\u6587'\n".encode(),
"line 3, column 11",
"'gbk' codec can't decode",
),
# UTF-8 by default, a comment included, which Python 3.12's parser passes.
(b"a = 1\nb = 2\n# caf\xe9\n", "line 3, column 6", "'utf-8' codec can't decode byte 0xe9"),
# After a BOM, which its codec strips before decoding the rest.
(b"\xef\xbb\xbfa = 1\nb = '\xff'\n", "line 2, column 6", "'utf-8' codec can't decode byte 0xff"),
# The two lines a declaration may take are read before anything is decoded.
(b"name = 'caf\xe9'\n", "line 1, column 1", "invalid or missing encoding declaration"),
(b"#!/usr/bin/env python\n# coding: nope\nx = 1\n", "line 2, column 1", "unknown encoding: nope"),
],
ids=["declared-encoding", "utf-8-comment", "after-a-bom", "first-line", "unknown-declaration"],
)
def test_bytes_python_cannot_decode_are_a_syntax_error_where_decoding_stops(tmp_path, raw, place, message):
from code_flow.tools.diagnostics import PythonDiagnostics
target = tmp_path / "undecodable.py"
target.write_bytes(raw)
note = _run(PythonDiagnostics().check(target, before=None))
assert f"WARNING: {place}: Python syntax error: {message}" in note
def test_a_finding_already_there_stays_existing_under_a_coding_declaration(registry, tmp_path):
"""The content an edit replaced is decoded as the content it wrote is, so
a finding on a line the declaration decodes the same way both times is
an existing finding, not a new one."""
target = tmp_path / "declared.py"
body = "# coding: latin-1\nlabel = 'caf\u00e9' + missing_one\nvalue = 1\n"
assert "missing_one" in _text(_run(registry.execute("write_file", {"file_path": str(target), "content": body})))
args = {"file_path": str(target), "old_string": "value = 1", "new_string": "value = 2"}
edited = _text(_run(registry.execute("edit_file", args)))
assert "Successfully edited" in edited and "missing_one" not in edited
def test_replaced_text_over_the_name_check_bound_in_bytes_is_an_unknown_baseline(tmp_path):
from code_flow.tools.diagnostics import MAX_SOURCE_BYTES, PythonDiagnostics
# Fewer characters than the bound, more bytes once stored.
before = "# " + "\u00e9" * (MAX_SOURCE_BYTES // 2)
assert len(before) <= MAX_SOURCE_BYTES < len(before.encode("utf-8"))
note = PythonDiagnostics().note(tmp_path / "wide.py", "value = missing_name\n", before=before)
assert "missing_name" in note and "baseline is unknown" in note
def test_text_is_checked_as_the_utf8_bytes_the_file_tool_stores(tmp_path):
"""Each of these compiles when the string alone is parsed and not as the
bytes on disk, which is what Python runs."""
from code_flow.tools.diagnostics import PythonDiagnostics
text = "# coding: ascii\nname = 'caf\u00e9'\n"
note = _run(PythonDiagnostics().check(tmp_path / "declared.py", text=text, before=None))
assert "line 2, column 12: Python syntax error: 'ascii' codec" in note
declared = "# coding: latin-1\nname = 'caf\u00e9'\nvalue = missing_name\n"
note = _run(PythonDiagnostics().check(tmp_path / "latin.py", text=declared, before=None))
assert "line 3, column 9: F821" in note
def test_a_checked_file_is_decoded_by_its_declaration_and_read_to_one_past_the_bound(tmp_path):
from code_flow.tools.diagnostics import MAX_SYNTAX_BYTES, PythonDiagnostics, read_source
target = tmp_path / "encoded.py"
target.write_bytes(b"# coding: latin-1\nname = 'caf\xe9'\nvalue = missing_name\n")
assert "WARNING: line 3, column 9: F821" in _run(PythonDiagnostics().check(target, before=None))
target.write_bytes(b"#" * (MAX_SYNTAX_BYTES + 5))
assert len(read_source(target)) == MAX_SYNTAX_BYTES + 1
assert "diagnostics skipped" in _run(PythonDiagnostics().check(target, before=None))
for absent in (tmp_path / "absent.py", tmp_path):
with pytest.raises(FileNotFoundError):
read_source(absent)
def test_a_baseline_is_read_as_bytes_and_bounded(tmp_path):
from code_flow.tools.diagnostics import MAX_SOURCE_BYTES, UNKNOWN, read_python
target = tmp_path / "encoded.py"
assert read_python(target) is None
raw = b"# coding: latin-1\nname = 'caf\xe9'\n"
target.write_bytes(raw)
assert read_python(target) == raw
target.write_bytes(b"x" * (MAX_SOURCE_BYTES + 1))
assert read_python(target) is UNKNOWN
assert read_python(tmp_path) is UNKNOWN
@pytest.mark.asyncio
async def test_a_baseline_read_runs_on_a_worker_and_gives_up_at_the_wait(tmp_path, monkeypatch):
"""The content an append replaces is read as every diagnostic read is, on
a diagnostic worker: a stalled read leaves the event loop free and the
baseline unknown once the wait runs out."""
import time
from code_flow.tools import diagnostics
release = threading.Event()
readers = []
def stalled(path):
readers.append(threading.get_ident())
release.wait(5)
return b""
monkeypatch.setattr(diagnostics, "read_python", stalled)
monkeypatch.setattr(diagnostics, "CHECK_WAIT_SECONDS", 0.05)
try:
started = time.monotonic()
assert await diagnostics.read_before(tmp_path / "slow.py") is diagnostics.UNKNOWN
assert time.monotonic() - started < 1
assert readers and readers[0] != threading.get_ident()
finally:
release.set()
@pytest.mark.parametrize("tool", ["write_file", "edit_file"])
def test_a_coding_declaration_applies_to_the_text_a_write_or_edit_stores(registry, tmp_path, tool):
"""The file tool stores its text as UTF-8, so a declaration in that text
governs the file Python reads: content the declaration cannot decode is
reported, as it is for the same bytes read back after an append."""
target = tmp_path / "declared.py"
body = "# coding: ascii\nname = 'x'\n"
if tool == "write_file":
args = {"file_path": str(target), "content": body.replace("'x'", "'caf\u00e9'")}
else:
assert "Successfully" in _text(
_run(registry.execute("write_file", {"file_path": str(target), "content": body}))
)
args = {"file_path": str(target), "old_string": "'x'", "new_string": "'caf\u00e9'"}
result = _text(_run(registry.execute(tool, args)))
assert "line 2, column 12: Python syntax error: 'ascii' codec can't decode byte 0xc3" in result
with pytest.raises(SyntaxError):
compile(target.read_bytes(), str(target), "exec")
def test_an_unavailable_note_names_its_file(tmp_path, monkeypatch):
"""Post-command checks append one note per changed file, so each
unavailable note says which file it is about, from a failed read and a
failed parse alike."""
from code_flow.tools import diagnostics
unreadable = tmp_path / "locked.py"
unreadable.write_text("value = 1\n")
deep = tmp_path / "deep.py"
deep.write_text("value = 1\n")
read = diagnostics.read_source
def refused(path):
if path == unreadable:
raise PermissionError(f"permission denied: {path}")
return read(path)
def too_deep(*args, **kwargs):
raise RecursionError("nested too deeply")
monkeypatch.setattr(diagnostics, "read_source", refused)
monkeypatch.setattr(diagnostics, "ast", SimpleNamespace(parse=too_deep))
read_note = _run(diagnostics.PythonDiagnostics().check(unreadable))
parse_note = _run(diagnostics.PythonDiagnostics().check(deep))
assert f"unavailable (PermissionError) for {unreadable};" in read_note
assert f"unavailable (RecursionError) for {deep};" in parse_note
@pytest.mark.asyncio
async def test_diagnostic_wait_is_bounded_and_cancellation_is_preserved(tmp_path, monkeypatch):
from code_flow.tools import diagnostics
from code_flow.tools.diagnostic_workers import DiagnosticWorkers
release = threading.Event()
def stalled(*args, **kwargs):
release.wait(2)
return ""
monkeypatch.setattr(diagnostics, "WORKERS", DiagnosticWorkers())
monkeypatch.setattr(diagnostics, "CHECK_WAIT_SECONDS", 0.01)
monkeypatch.setattr(diagnostics.PythonDiagnostics, "note", stalled)
try:
note = await diagnostics.PythonDiagnostics().check(tmp_path / "slow.py", text="value = 1\n")
assert "diagnostics unavailable (TimeoutError)" in note
task = asyncio.create_task(diagnostics.PythonDiagnostics().check(tmp_path / "cancelled.py", text="value = 1\n"))
task.cancel()
with pytest.raises(asyncio.CancelledError):
await task
finally:
release.set()
# --- occurrence -----------------------------------------------------------------------
def test_occurrence_replaces_exactly_the_nth_match(registry, tmp_path):
target = tmp_path / "i.txt"
target.write_text("a\na\na\n")
_run(registry.execute("read_file", {"file_path": str(target)}))
result = _run(
registry.execute("edit_file", {"file_path": str(target), "old_string": "a", "new_string": "b", "occurrence": 2})
)
assert "occurrence 2 of 3" in _text(result) and "line 2" in _text(result)
assert target.read_text() == "a\nb\na\n"
direct = _run(
registry.get("edit_file").execute(file_path=str(target), old_string="a", new_string="c", occurrence=2)
)
assert isinstance(direct, ToolResult) and direct.file_change is not None and direct.diff
assert target.read_text() == "a\nb\nc\n"
@pytest.mark.parametrize(
("content", "old", "occurrence", "expected"),
[
("aaaa bb aa", "aa", 1, "Xaa bb aa"),
("aaaa bb aa", "aa", 2, "aaX bb aa"),
("aaaa bb aa", "aa", 3, "aaaa bb X"),
("ababa aba", "aba", 2, "ababa X"),
("aa\r\naa\r\n", "aa\r\n", 2, "aa\r\nX"),
],
)
def test_occurrence_counts_non_overlapping_matches(registry, tmp_path, content, old, occurrence, expected):
target = tmp_path / "matches.txt"
target.write_bytes(content.encode())
_run(registry.execute("read_file", {"file_path": str(target)}))
result = _run(
registry.execute(
"edit_file",
{"file_path": str(target), "old_string": old, "new_string": "X", "occurrence": occurrence},
)
)
assert not _text(result).startswith("Error")
assert target.read_bytes() == expected.encode()
def test_occurrence_description_states_its_matching_rules(registry):
tool = registry.get("edit_file")
description = tool.parameters["properties"]["occurrence"]["description"]
assert "non-overlapping" in description and "1-based" in description
assert "exact" in description and "replace_all" in description
assert "non-overlapping" in tool.description
def test_occurrence_past_the_count_and_with_replace_all_are_refused(registry, tmp_path):
target = tmp_path / "j.txt"
target.write_text("a\na\n")
_run(registry.execute("read_file", {"file_path": str(target)}))
too_far = _text(
_run(
registry.execute(
"edit_file", {"file_path": str(target), "old_string": "a", "new_string": "b", "occurrence": 3}
)
)
)
assert too_far.startswith("Error") and "matches only 2" in too_far
both = _text(
_run(
registry.execute(
"edit_file",
{"file_path": str(target), "old_string": "a", "new_string": "b", "occurrence": 1, "replace_all": True},
)
)
)
assert both.startswith("Error") and "mutually exclusive" in both
assert target.read_text() == "a\na\n"
def test_several_matches_without_a_selector_are_still_refused(registry, tmp_path):
target = tmp_path / "k.txt"
target.write_text("a\na\n")
_run(registry.execute("read_file", {"file_path": str(target)}))
result = _text(
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "a", "new_string": "b"}))
)
assert "appears 2 times" in result and "old_string" in result and "old_text" not in result
assert target.read_text() == "a\na\n"
# --- glob ------------------------------------------------------------------------------
def test_glob_finds_by_pattern_and_expands_a_brace_alternation(tmp_path):
(tmp_path / "one.py").write_text("")
(tmp_path / "two.txt").write_text("")
(tmp_path / "three.md").write_text("")
tool = GlobTool(workspace=tmp_path)
with workdir.bind(tmp_path):
py = _text(_run(tool.execute(pattern="*.py")))
assert "one.py" in py and "two.txt" not in py
both = _text(_run(tool.execute(pattern="*.{py,txt}")))
assert "one.py" in both and "two.txt" in both and "three.md" not in both
def test_brace_expansion_rules():
assert expand_braces("*.{a,b}") == ["*.a", "*.b"]
assert expand_braces("src/{a,b}/*.{ts,tsx}") == ["src/a/*.ts", "src/a/*.tsx", "src/b/*.ts", "src/b/*.tsx"]
assert expand_braces("*.py") == ["*.py"]
# --- the replacement in the registry -----------------------------------------------------
def test_a_plugin_tool_registered_after_the_builtin_replaces_it(tmp_path):
reg = ToolRegistry()
reg.register(TrunkReadFileTool(workspace=tmp_path))
assert reg.get("read_file").parameters["required"] == ["path"]
reg.register(ReadFileTool(workspace=tmp_path))
assert reg.get("read_file").parameters["required"] == ["file_path"]
assert isinstance(reg.get("read_file"), ReadFileTool)
def test_the_ledger_is_keyed_by_resolved_path(tmp_path):
ledger = ReadLedger()
target = tmp_path / "m.txt"
target.write_text("z\n")
read = ReadFileTool(workspace=tmp_path, ledger=ledger)
edit = EditFileTool(workspace=tmp_path, ledger=ledger)
with workdir.bind(tmp_path):
_run(read.execute(file_path="m.txt"))
assert ledger.has_read(target)
assert "Successfully edited" in _text(_run(edit.execute(file_path=str(target), old_string="z", new_string="w")))
def test_write_file_still_reports_the_diff_the_ui_renders(tmp_path):
tool = WriteFileTool(workspace=tmp_path)
target = tmp_path / "n.py"
target.write_text("a = 1\n")
result = _run(tool.execute(file_path=str(target), content="a = 2\n"))
assert isinstance(result, ToolResult) and result.file_change is not None
def _git(cwd: Path, *args: str) -> None:
subprocess.run(["git", "-C", str(cwd), *args], check=True, capture_output=True)
# --- what the adversarial audit of 2026-09-08 found, each pinned -------------
def test_an_occurrence_edit_keeps_the_files_own_line_endings(registry, tmp_path):
"""A CRLF file edited through ``occurrence`` used to come back with every
line changed: the path read and wrote through text mode, which normalises
to LF, so a one-line edit arrived as a whole-file diff (and a grader diffing
the working tree sees a rewrite). Trunk's own edit path reads and writes
bytes for exactly this reason; so does this one now."""
target = tmp_path / "crlf.txt"
target.write_bytes(b"a\r\nb\r\na\r\n")
_run(registry.execute("read_file", {"file_path": str(target)}))
result = _run(
registry.execute("edit_file", {"file_path": str(target), "old_string": "a", "new_string": "Z", "occurrence": 2})
)
assert "occurrence 2 of 2" in _text(result)
assert target.read_bytes() == b"a\r\nb\r\nZ\r\n"
def test_an_error_report_quotes_the_file_verbatim(registry, tmp_path):
"""The respelling of trunk's parameter names must not reach quoted file
content. It used to be a blanket replacement over the whole result, so a
file that really contains ``old_text`` was shown to the model as containing
``old_string`` -- and the model then copied an excerpt that could never
match anything."""
target = tmp_path / "tool.py"
target.write_text("def execute(self, path, old_text, new_text):\n value = old_text\n return path\n")
_run(registry.execute("read_file", {"file_path": str(target)}))
near_miss = "def execute(self, path, old_text, new_text):\n value = old_text\n return other\n"
report = _text(
_run(
registry.execute(
"edit_file",
{"file_path": str(target), "old_string": near_miss, "new_string": "x"},
)
)
)
sentence, _, excerpt = report.partition("\n")
assert sentence.startswith("Error") and "old_string" in sentence, "the sentence names the parameter shown"
assert "old_text" in excerpt, "the quoted file keeps its own spelling"
assert "old_string" not in excerpt, "and nothing rewrites the file's text"
def test_a_long_line_is_cut_the_way_the_description_promises(registry, tmp_path):
"""Trunk numbers a line whole, so one minified line can overrun the
whole-result budget by itself: the read came back with NO body and a note
telling the model to continue from the same offset, which it can do
forever. The description promised truncation; now it happens."""
target = tmp_path / "bundle.js"
target.write_text("x" * 200_000 + "\nlast\n")
body = _text(_run(registry.execute("read_file", {"file_path": str(target)})))
assert "line truncated to 2000 chars" in body
assert "1| " in body and "2| last" in body
def test_minified_lines_are_cut_before_the_page_is_counted_so_they_share_one_page(registry, tmp_path):
"""Fifty lines of 10k characters: cut after trunk's page, a dozen made a
page; cut on trunk's hook, before its budget counts them, they all do, each
cut once (the cut after the read is only for a page a registered
machine's node answered)."""
target = tmp_path / "chunks.js"
target.write_text("\n".join("x" * 10_000 for _ in range(50)) + "\n")
body = _text(_run(registry.execute("read_file", {"file_path": str(target)})))
lines = body.split("\n\n")[0].split("\n")
assert len(lines) == 50 and lines[0].startswith("1| ") and lines[-1].startswith("50| ")
assert lines[0] == "1| " + "x" * 2000 + "... (line truncated to 2000 chars)", "cut once, at the budget"
assert body.endswith("(End of file \u2014 50 lines total)")
assert "(Line 1 is" not in body
def test_a_read_that_failed_does_not_license_an_edit(registry, tmp_path):
"""A model probing for a file that does not exist yet used to have that
failed read recorded as having seen it; once something else created the
file, the edit went straight through on content nobody had read."""
target = tmp_path / "later.txt"
missing = _text(_run(registry.execute("read_file", {"file_path": str(target)})))
assert missing.startswith("Error")
target.write_text("created by something else\n")
refused = _text(
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "created", "new_string": "x"}))
)
assert refused.startswith("Error") and "have not read" in refused
def test_a_file_that_changed_since_the_read_is_refused_as_stale(registry, tmp_path):
"""The fork's third state: a formatter, a build step or a checkout between
the read and the edit means the text being edited is no longer there."""
import os
target = tmp_path / "moving.txt"
target.write_text("one\n")
_run(registry.execute("read_file", {"file_path": str(target)}))
target.write_text("two\n")
os.utime(target, (1_000_000, 1_000_000))
stale = _text(
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "two", "new_string": "three"}))
)
assert stale.startswith("Error") and "changed since you last read it" in stale
_run(registry.execute("read_file", {"file_path": str(target)}))
assert "Successfully edited" in _text(
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "two", "new_string": "three"}))
)
def test_the_write_ledger_does_not_depend_on_the_syntax_verdict(tmp_path):
"""Two separate knobs: turning the Python verdict off used to leave the
model unable to edit a file it had just written, because the ledger entry
sat after the verdict's early return."""
ctx = _ctx(tmp_path, {"enabled": True, "pythonSyntaxNote": False})
write = factories.make_write_file(ctx)
edit = factories.make_edit_file(ctx)
target = tmp_path / "quiet.py"
assert "Successfully wrote" in _text(_run(write.execute(file_path=str(target), content="x = 1\n")))
assert "Successfully edited" in _text(_run(edit.execute(file_path=str(target), old_string="1", new_string="2")))
def test_the_workspace_fence_follows_the_products_answer(tmp_path):
"""These tools replace the host's own by name, and the host fences its own
on ``tools.restrictToWorkspace``. Passing no ``allowed_dirs`` meant the
fence was OFF whatever the product asked for -- half the face fenced, half
not, with no warning. The launcher renders the answer into the slice."""
outside = tmp_path / "outside.txt"
outside.write_text("secret\n")
inside = tmp_path / "ws"
inside.mkdir()
unfenced = factories.make_read_file(_ctx(inside, {"enabled": True}))
assert "secret" in _text(_run(unfenced.execute(file_path=str(outside))))
fenced = factories.make_read_file(_ctx(inside, {"enabled": True, "restrictToWorkspace": True}))
refused = _text(_run(fenced.execute(file_path=str(outside))))
assert refused.startswith("Error") and "outside allowed" in refused
def test_a_zero_or_negative_occurrence_is_refused_in_code(tmp_path):
"""The schema's ``minimum`` stops the model, but a caller reaching the tool
directly used to get a write at a negative index reported as success."""
tool = factories.make_edit_file(_ctx(tmp_path, {"enabled": True, "requireReadBeforeEdit": False}))
target = tmp_path / "guard.txt"
target.write_text("aaa\n")
for bad in (0, -1):
refused = _text(_run(tool.execute(file_path=str(target), old_string="a", new_string="Z", occurrence=bad)))
assert refused.startswith("Error") and "1 or greater" in refused
assert target.read_text() == "aaa\n"
# --- glob: the merge that used to paste the engine's notices into the listing --
def test_glob_merges_only_paths_and_honours_the_callers_limit(tmp_path):
"""Every alternative used to spend the caller's ``limit`` again, and each
engine notice ("No files found…", "(showing first N of M results)") was
pasted into the listing as though it were a filename -- which the model
then tried to read."""
for name in ("a.py", "b.py", "c.ts", "d.ts"):
(tmp_path / name).write_text("")
tool = GlobTool(workspace=tmp_path)
with workdir.bind(tmp_path):
both = _text(_run(tool.execute(pattern="*.{py,ts}")))
assert sorted(both.splitlines()) == ["a.py", "b.py", "c.ts", "d.ts"]
missing = _text(_run(tool.execute(pattern="*.{py,zzz}")))
assert sorted(missing.splitlines()) == ["a.py", "b.py"], "a barren alternative adds no line"
capped = _text(_run(tool.execute(pattern="*.{py,ts}", limit=3)))
lines = capped.splitlines()
assert len(lines) == 4 and len([ln for ln in lines if is_notice(ln)]) == 1
assert len([ln for ln in lines if not is_notice(ln)]) == 3, "the caller's limit is the merged total"
# A single pattern is the engine's own answer, verbatim.
assert "No files found matching pattern" in _text(_run(tool.execute(pattern="*.nope")))
# A merged one is this tool's, and names the pattern it was asked for.
assert "No files found matching pattern: *.{nope,zzz}" == _text(_run(tool.execute(pattern="*.{nope,zzz}")))
def test_glob_preserves_a_child_truncation_when_other_patterns_are_empty(tmp_path):
for name in ("a.py", "b.py", "c.py"):
(tmp_path / name).write_text("")
tool = GlobTool(workspace=tmp_path)
out = _text(_run(tool.execute(pattern="*.{py,ts}", limit=1)))
assert "PARTIAL" in out and "raise limit" in out
assert len([line for line in out.splitlines() if not is_notice(line)]) == 1
def test_glob_carries_a_childs_traversal_budget_into_the_merged_notice(tmp_path, monkeypatch):
"""When the engine's walk hits its deadline the child answer ends in a
PARTIAL trailer, sometimes above an otherwise clean miss. Neither line is a
path, and the merged listing has to say the search was cut short rather
than report a tree that lacks the files."""
from raven.agent.tools import tree_walk
(tmp_path / "a.py").write_text("")
tool = GlobTool(workspace=tmp_path)
monkeypatch.setattr(tree_walk, "WALK_DEADLINE_S", -1.0)
merged = _text(_run(tool.execute(pattern="*.{py,ts}")))
assert merged.startswith("No files found matching pattern: *.{py,ts}")
assert "traversal budget" in merged
assert all(is_notice(line) for line in merged.splitlines() if line)
# Each child walk reads the clock four times over this tree: its
# deadline, then before ``late/`` and ``a.py`` at the root (in budget),
# then before ``late/b.ts`` (past it).
monkeypatch.setattr(tree_walk, "WALK_DEADLINE_S", 20.0)
ticks = itertools.cycle([0.0, 0.0, 0.0, tree_walk.WALK_DEADLINE_S + 1])
monkeypatch.setattr(tree_walk, "time", SimpleNamespace(monotonic=lambda: next(ticks)))
(tmp_path / "late").mkdir()
(tmp_path / "late" / "b.ts").write_text("")
partial = _text(_run(tool.execute(pattern="*.{py,ts}")))
lines = partial.splitlines()
assert [line for line in lines if not is_notice(line)] == ["a.py"]
assert "PARTIAL result" in lines[-1] and "traversal budget" in lines[-1]
@pytest.mark.parametrize("kind", ["missing", "not_directory", "outside"])
def test_glob_preserves_child_search_errors(tmp_path, kind):
root = tmp_path / "root"
root.mkdir()
target = tmp_path / "outside" if kind == "outside" else root / kind
if kind == "not_directory":
target.write_text("file")
elif kind == "outside":
target.mkdir()
tool = GlobTool(workspace=root, allowed_dirs=(root,))
single = _text(_run(tool.execute(pattern="*.py", path=str(target))))
merged = _text(_run(tool.execute(pattern="*.{py,ts}", path=str(target))))
assert single.startswith("Error:") and merged == single
def test_glob_merges_by_recency_before_applying_the_total_limit(tmp_path):
for timestamp, name in enumerate(("old.py", "middle.py", "new.ts"), start=100):
path = tmp_path / name
path.write_text("")
os.utime(path, (timestamp, timestamp))
tool = GlobTool(workspace=tmp_path)
out = _text(_run(tool.execute(pattern="*.{py,ts}", limit=2)))
assert out.splitlines()[:2] == ["new.ts", "middle.py"]
assert "PARTIAL" in out
def test_glob_keeps_paths_that_look_like_notice_prefixes_and_deduplicates(tmp_path):
names = ["ErrorHandler.py", "No files found matching pattern.py", "other.py"]
for name in names:
(tmp_path / name).write_text("")
tool = GlobTool(workspace=tmp_path)
out = _text(_run(tool.execute(pattern="{*.py,Error*}", limit=3)))
assert sorted(out.splitlines()) == sorted(names)
assert "PARTIAL" not in out
def test_nested_braces_expand_by_pairing_and_a_runaway_pattern_is_refused(tmp_path):
"""Scanning for the first ``}`` produced garbage patterns (``*.a}``) and
silently dropped whole alternatives -- a listing that omits half the
alternatives reads exactly like a tree that lacks them."""
assert expand_braces("*.{a,b}") == ["*.a", "*.b"]
assert expand_braces("*.{a,{b,c}}") == ["*.a", "*.b", "*.c"]
assert expand_braces("src/{a,b}/*.{ts,tsx}") == ["src/a/*.ts", "src/a/*.tsx", "src/b/*.ts", "src/b/*.tsx"]
assert expand_braces("*.py") == ["*.py"]
with pytest.raises(ValueError):
expand_braces("*.{a,b")
with pytest.raises(ValueError):
expand_braces("{a,b,c,d,e}{1,2,3,4,5}")
(tmp_path / "one.py").write_text("")
tool = GlobTool(workspace=tmp_path)
with workdir.bind(tmp_path):
refused = _text(_run(tool.execute(pattern="{a,b,c,d,e}{1,2,3,4,5}")))
assert refused.startswith("Error") and "separate calls" in refused
assert _text(_run(tool.execute(pattern="*.{py,{txt,md}}"))).splitlines() == ["one.py"]
# --- the boundary the same-name contribution stands on ------------------------
def test_the_registered_instance_is_this_products_under_every_shared_name(tmp_path):
"""The guarantee the whole face rests on: where a name is shared with the
host, the object the registry dispatches to is THIS product's.
Checked by identity, not by schema: a change that registers the plugin
tools before the built-ins instead of after would leave every name in
place and every schema silently reverted to the host's spelling."""
from raven.agent.tools import file_search as trunk_search
from raven.agent.tools import filesystem as trunk_fs
ctx = _ctx(tmp_path, {"enabled": True})
reg = ToolRegistry()
# The host's own order: built-ins first, plugin contributions last.
for cls in (
trunk_fs.ReadFileTool,
trunk_fs.WriteFileTool,
trunk_fs.EditFileTool,
trunk_fs.ListDirTool,
trunk_search.GrepTool,
trunk_search.FindTool,
):
reg.register(cls(workspace=tmp_path))
for make in (
factories.make_read_file,
factories.make_write_file,
factories.make_edit_file,
factories.make_list_dir,
factories.make_glob,
factories.make_todo,
factories.make_exec,
):
reg.register(make(ctx))
for name in ("read_file", "write_file", "edit_file", "list_dir"):
served = reg.get(name)
assert type(served).__module__.startswith("code_flow.tools"), name
assert not isinstance(served, (trunk_fs.ReadFileTool,)) or type(served) is not trunk_fs.ReadFileTool
# Names the host does not use come from here too, and the host's own
# spelling of the pathname tool is still registered until the launcher
# withholds it -- that withholding is the launcher's test, not this one.
assert type(reg.get("glob")).__module__.startswith("code_flow.tools")
assert type(reg.get("todo")).__module__.startswith("code_flow.tools")
assert type(reg.get("grep")).__module__.startswith("raven."), "a name we do not serve stays the host's"
assert reg.get("find") is not None, "trunk's spelling is withheld by config, not by this registry"
def test_the_replacement_answers_the_call_not_just_the_schema(tmp_path):
"""Dispatch, not advertising: the served instance is the one that runs, so
a call in the fork's spelling reaches this product's behaviour (the read
ledger) rather than the host's tool, which has none."""
ctx = _ctx(tmp_path, {"enabled": True})
reg = ToolRegistry()
reg.register(trunk_read_file(tmp_path))
reg.register(factories.make_read_file(ctx))
reg.register(factories.make_edit_file(ctx))
target = tmp_path / "b.txt"
target.write_text("one\n")
# The host's tool has no read-before-edit rule; ours does, and it is ours
# that answers.
refused = _text(
_run(registry_execute(reg, "edit_file", {"file_path": str(target), "old_string": "one", "new_string": "two"}))
)
assert refused.startswith("Error") and "have not read" in refused
_run(registry_execute(reg, "read_file", {"file_path": str(target)}))
assert "Successfully edited" in _text(
_run(registry_execute(reg, "edit_file", {"file_path": str(target), "old_string": "one", "new_string": "two"}))
)
def test_the_face_declines_as_a_whole_and_the_host_keeps_serving(tmp_path):
"""A product that hands the tools back gets the host's, unchanged: the
factories decline, nothing is registered over the built-ins, and the
schema the model sees is the host's own."""
from raven.agent.tools import filesystem as trunk_fs
ctx = PluginContext(
config={"enabled": True, "tools": {"enabled": False}},
services=ServiceLocator(workspace=tmp_path, user_id="u", agent_id="a"),
)
reg = ToolRegistry()
reg.register(trunk_fs.ReadFileTool(workspace=tmp_path))
contributed = [
make(ctx)
for make in (
factories.make_read_file,
factories.make_write_file,
factories.make_edit_file,
factories.make_list_dir,
factories.make_glob,
factories.make_todo,
factories.make_exec,
)
]
assert contributed == [None] * 7
assert type(reg.get("read_file")) is trunk_fs.ReadFileTool
assert reg.get("read_file").parameters["required"] == ["path"]
def test_a_malformed_slice_declines_rather_than_serving_half_a_face(tmp_path):
"""Half a face is worse than none: the guide and the conduct teach one
spelling, so a slice that does not parse must take the whole face down."""
ctx = PluginContext(
config={"enabled": True, "tools": {"enabled": "yes please"}},
services=ServiceLocator(workspace=tmp_path, user_id="u", agent_id="a"),
)
assert factories.make_read_file(ctx) is None
assert factories.make_glob(ctx) is None
_MARK = "... (line truncated to 2000 chars)"
def test_a_line_whose_own_text_ends_like_the_mark_is_cut_like_any_other(registry, tmp_path):
"""A 5000-character line ending the way a cut line does came back whole: the
pass after the read took the ending for its own mark (found by both
reviewers of the first head). Whether a line was cut is decided by where
the read ran, never by the line's text."""
target = tmp_path / "pasted.txt"
target.write_text("x" * (5_000 - len(_MARK)) + _MARK + "\nlast\n")
body = _text(_run(registry.execute("read_file", {"file_path": str(target)})))
lines = body.split("\n")
assert lines[0] == "1| " + "x" * 2000 + _MARK and lines[1] == "2| last"
def test_a_page_a_registered_machine_s_node_answered_is_cut_after_the_read(tmp_path, monkeypatch):
"""The node runs trunk's tool, without this fork's hook, so its page is cut here, line by line, once."""
import json
from raven.ops import remote_path
rows = [
{"id": "box", "display_name": "Lab box", "host": "203.0.113.7", "port": 58717, "user": "worker", "key": "/k"}
]
(tmp_path / "connections.json").write_text(json.dumps({"connections": rows}), encoding="utf-8")
monkeypatch.setenv("RAVEN_CONNECTIONS", str(tmp_path / "connections.json"))
monkeypatch.setattr(remote_path, "_HOMES", {})
monkeypatch.setattr(remote_path, "_NODES", None)
monkeypatch.setattr(remote_path, "_READ", {})
monkeypatch.setattr(
"raven.config.loader.load_config", lambda *a, **k: SimpleNamespace(subagents=SimpleNamespace(agents=[]))
)
class Pool:
async def home(self, machine_id: str) -> str:
return "/home/worker"
async def call(self, machine_id: str, method: str, params: dict) -> dict:
assert method == "fs/read"
return {"text": "1| " + "y" * 5_000 + "\n2| last\n\n(End of file — 2 lines total)", "sha256": "ab" * 32}
remote_path.use_nodes(Pool)
read = ReadFileTool(workspace=tmp_path, allowed_dirs=(tmp_path,), follow_binding=False)
got = _run(read.execute(file_path="box:raven-work/t1/bundle.js"))
lines = got.split("\n")
assert lines[0] == ("1| " + "y" * 5_000)[:2000] + _MARK and lines[1] == "2| last"
assert got.endswith("(End of file — 2 lines total)")