## Summary On Windows, the file tools took a DOS device name for an ordinary workspace file: `write_file` with `path="nul"` sent the content to the NUL device and answered success for a file that never existed, `diff` and `file_change` included. The same holds for the rest of the reserved set under Windows' naming rules: a first extension and trailing spaces are stripped before the device comparison, so `nul.md` and `con ` are devices too, and on Windows 10 and earlier the whole set is reserved in every directory. Root cause: `resolve_path` (`raven/agent/tools/filesystem.py:57-76`) joins the name onto the workspace and calls `Path.resolve()`. On Windows that is `ntpath.realpath`, which maps only a path that is exactly `nul` to `\\.\NUL` and hands `<workspace>\nul` back unchanged, so the `allowed_dirs` prefix check then accepts it, and the file has no reserved-name check anywhere. CPython tracks the general "device name inside a longer path" gap as issue 95486. Fix: `resolve_path` refuses the reserved shapes #895 names, in every component the caller supplies -- a device name with or without a first extension, the superscript COM/LPT aliases, a name ending in a dot or a space, and a colon stream. The set and the per-component order mirror CPython 3.13's `ntpath.isreserved`, which Python 3.12 cannot call; Microsoft's naming rules list COM/LPT 1-9 only, so `com0.txt` is an ordinary file and stays one. The refusal is a `PermissionError`, the same one a path outside the allowed directories gets, and the guard is gated on `_WINDOWS` (snapshotted from `os.name == "nt"` at import). Every consumer reaches it through this single choke point -- the five file tools, `deliver_files`, the RPC file viewer and the turn attachments -- so one check covers them all. A URL argument keeps its `web_fetch` hint, and a drive letter written with double slashes stays inside the check, since Windows reads it as a local path. Two review rounds also covered the remote-path layer these tools dispatch to. Only the head of an scp spelling -- the machine id or `user@address`, before the separator -- can be the device instead of a machine, so `NUL:` and `com1:x` take the fence first while `box:raven-work/a.txt` and files named `nul`, `aux` or `com1` on that machine reach it. The classification asks the remote-path parser whether the spelling is remote, so the two layers read one rule rather than a second parser. Off Windows nothing changes: a file called `nul` stays a legal name, and the tests cover both sides of the gate. The branch history carries the commits this change was developed in; the latest commit migrates the fixes reviewed on the replacement branch, and the author's own commits are preserved as they were. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification All commands run with `timeout`, on this branch's head with a clean tree, against the repo `.venv`. - Red first (tree before the fix): inline `timeout 60 .venv/bin/python` calling `resolve_path` for `nul` / `nul.md` / `con.md` / `report.md` -> all four resolved as ordinary paths (device names accepted); `pytest tests/test_filesystem_tools.py` -> ImportError (`_is_device_name` absent). - Green, gate on (`os.name == "nt"` simulated): same inline script -> `nul`, `nul.md` and `con.md` raise `PermissionError`, `report.md` resolves normally. - Gate off (POSIX default): same inline script -> all four names resolve as before. - File: `timeout 120 .venv/bin/python -m pytest tests/test_filesystem_tools.py -q -p no:cacheprovider` -> 54 passed. - Related subset: `timeout 300 .venv/bin/python -m pytest tests/ -k "filesystem or file_tool or fence" -q -p no:cacheprovider` -> 281 passed, 7 skipped. - Direct consumers of `resolve_path` (5 test files): 288 passed, 1 failed (`test_a_host_without_libreoffice_says_so`; reproduced identical red on a clean tree -> pre-existing environment failure). - Full suite: `timeout 590 .venv/bin/python -m pytest tests -q --ignore=tests/manual -p no:cacheprovider` -> 27973 passed, 128 skipped, 4 failed (a2a interop / rpc_files libreoffice / simulation suite / simulation scenario; each reproduced identical red on clean `origin/main` -> pre-existing environment failures, not new regressions). - Lint: `timeout 120 uvx ruff@latest check raven/` -> All checks passed; `uv run ruff format` applied (touches the added lines only), `ruff check` -> All checks passed. - At the current head `6700103ad`, in a worktree cut from this branch, with the repo's Python 3.12 environment (`--all-extras`) and an isolated interpreter whose site-packages put this worktree's `raven` first (the imported `raven` was confirmed by printing its `__file__`): - `pytest tests/test_filesystem_tools.py -q -n 0` -> 133 passed. - `pytest` over the eleven files that reach `resolve_path`, the remote answer or the node client (`test_filesystem_tools`, `test_filesystem_write_modes`, `test_file_search_traversal_guard`, `test_deliver_files_tool`, `test_rpc_files`, `test_rpc_turn_send`, `test_ops_remote_path`, `test_permissions_remote_read`, `test_agents_code_tools_plugin`, `test_remote_file_tools`, `test_node_client`) -> 782 passed, including the real-node cases behind the stand-in ssh. - Red first, latest round: with the previous head's source restored, 37 tests fail -- every remote reserved-name case for all four tools, plus the local control -- and with the fix all 133 pass. Round one's evidence also holds: the earlier broad predicate fails the remote-path case, and disabling the device check fails the colon spellings. - The remote cases are parametrized over `read_file`, `write_file`, `edit_file` and `list_dir`, over scp short forms, `user@host`, absolute and relative remote paths and the `scp://` form (`nul`, `aux`, `com1`, `nul.md`, `nul:hidden` included). Each asserts the remote answer received that exact path and that no local directory entry was written; the control asserts local reserved shapes are refused with no remote call. - Gate on, with the real tools: `nul`, `nul.md`, `con.md`, `notes.md.`, `notes.md ` and `subdir/nul` answer the refusal; `report.md`, `com0.txt` and `https://example.com/a.md` stay ordinary; gate off, `nul` resolves. - `ruff format --check` and `ruff check` on both files: clean. - `ty check raven/agent/tools/filesystem.py`: passes. - `lint-imports`: 10 contracts kept, 0 broken. - `scripts/check_source_language.py`, `scripts/check_commit_messages.py` and `scripts/check_large_files.py` over `origin/main...HEAD`: exit 0. - `git diff --check`: clean. - The real-node cases in `test_remote_file_tools.py` compare the node subprocess's package digest with the caller's, so both sides must import this branch: they pass in the isolated interpreter above (25 cases). Under a plain `PYTHONPATH` over a checkout whose installed tree differs, they fail for that digest reason alone. - Merge compatibility: a trial merge of this branch onto current `main` (`2742cf6c0`) completes with no conflict, touching the same two files. Not run: a real Windows end-to-end pass (development host is macOS, no Windows environment available). The refusal is gated on `_WINDOWS` and was verified at the logic level on both sides of the gate; the Windows-side behaviour rests on the CPython `ntpath` source (read at 3.13) and issue 95486, not on a live run. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No docs change: no document describes the file tools' path handling at this level. ## Risk User-visible change is Windows-only: the file tools now refuse a reserved name instead of silently opening the device, and a remote path reaches its machine whatever its components are named, including `nul`, `aux` and `com1`. Off Windows the gate is off and behaviour is unchanged. - [x] Security impact considered The change closes a silent write-to-device hole (content discarded, fake success receipt); it adds a refusal, removes no capability, and cannot widen any path. - [x] Backward compatibility considered POSIX filenames like `nul` or `nul.md` stay legal; on Windows such names never addressed a real file through these tools, so no working workflow breaks. - [x] Rollback path is clear for risky changes Rollback is a revert of the added commits; the guard is one string check at one fence point with no persistent state. ## Related Issues Fixes #895 --------- Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn>
795 lines
30 KiB
Python
795 lines
30 KiB
Python
"""Filesystem tools: the unified diff a write reports for the UI.
|
|
|
|
A whole-file write is the only change no UI can reconstruct afterwards -- the
|
|
previous content is gone the moment it lands -- so the tool has to hand it over.
|
|
"""
|
|
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
|
|
from raven.agent.tools import filesystem, remote_files
|
|
from raven.agent.tools.filesystem import (
|
|
EditFileTool,
|
|
ListDirTool,
|
|
ReadFileTool,
|
|
WriteFileTool,
|
|
_is_reserved_windows_component,
|
|
resolve_path,
|
|
)
|
|
from raven.agent.tools.registry import ToolRegistry
|
|
from raven.contracts.tool import ToolOutput, ToolResult
|
|
from raven.ops import remote_path
|
|
|
|
|
|
def _diff_of(result: ToolResult | str) -> str | None:
|
|
assert isinstance(result, ToolResult)
|
|
return result.diff
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_overwriting_a_file_reports_both_sides_of_the_change(tmp_path: Path):
|
|
target = tmp_path / "letter.md"
|
|
target.write_text("dear friend\nold line\nbye\n", encoding="utf-8")
|
|
|
|
out = await WriteFileTool(workspace=tmp_path).execute(path=str(target), content="dear friend\nnew line\nbye\n")
|
|
|
|
diff = _diff_of(out)
|
|
assert diff is not None
|
|
assert "-old line" in diff
|
|
assert "+new line" in diff
|
|
# The unchanged lines ride along as context, not as additions.
|
|
assert "+dear friend" not in diff
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_new_file_is_all_additions(tmp_path: Path):
|
|
out = await WriteFileTool(workspace=tmp_path).execute(path=str(tmp_path / "fresh.txt"), content="one\ntwo\n")
|
|
|
|
diff = _diff_of(out)
|
|
assert diff is not None
|
|
assert "+one" in diff and "+two" in diff
|
|
assert "\n-" not in diff
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_rewriting_identical_content_reports_no_diff(tmp_path: Path):
|
|
target = tmp_path / "same.txt"
|
|
target.write_text("unchanged\n", encoding="utf-8")
|
|
|
|
out = await WriteFileTool(workspace=tmp_path).execute(path=str(target), content="unchanged\n")
|
|
|
|
assert _diff_of(out) is None
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_rewrite_too_large_to_render_is_dropped_whole(tmp_path: Path):
|
|
target = tmp_path / "big.txt"
|
|
target.write_text("\n".join(f"old {i}" for i in range(500)), encoding="utf-8")
|
|
|
|
out = await WriteFileTool(workspace=tmp_path).execute(
|
|
path=str(target), content="\n".join(f"new {i}" for i in range(500))
|
|
)
|
|
|
|
# Half a diff would be worse than none, so nothing is sent.
|
|
assert _diff_of(out) is None
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_the_model_text_of_a_write_is_unchanged_by_the_diff(tmp_path: Path):
|
|
target = tmp_path / "note.txt"
|
|
out = await WriteFileTool(workspace=tmp_path).execute(path=str(target), content="hi\n")
|
|
|
|
assert isinstance(out, ToolResult)
|
|
assert out.model_text.startswith("Successfully wrote 3 bytes to ")
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_an_edit_reports_the_replacement_in_context(tmp_path: Path):
|
|
target = tmp_path / "code.py"
|
|
target.write_text("a = 1\nb = 2\nc = 3\n", encoding="utf-8")
|
|
|
|
out = await EditFileTool(workspace=tmp_path).execute(path=str(target), old_text="b = 2", new_text="b = 22")
|
|
|
|
diff = _diff_of(out)
|
|
assert diff is not None
|
|
assert "-b = 2" in diff and "+b = 22" in diff
|
|
assert " a = 1" in diff
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_the_registry_carries_the_diff_to_the_agent_loop(tmp_path: Path):
|
|
target = tmp_path / "letter.md"
|
|
target.write_text("before\n", encoding="utf-8")
|
|
registry = ToolRegistry()
|
|
registry.register(WriteFileTool(workspace=tmp_path))
|
|
|
|
out = await registry.execute("write_file", {"path": str(target), "content": "after\n"})
|
|
|
|
assert isinstance(out, ToolOutput)
|
|
assert out.diff is not None and "+after" in out.diff
|
|
# It rides beside the model text, never inside it.
|
|
assert "after" not in str(out).replace(str(target), "")
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_url_is_refused_by_naming_the_tool_that_can_reach_it(tmp_path: Path):
|
|
""" "File not found" describes the wrong problem for a URL.
|
|
|
|
The file is not missing; the argument belongs to another tool. Reported as a
|
|
missing file it reads as a misspelling, so the caller tries the same URL again
|
|
with the workspace prefixed, percent-decoded, a directory up -- none of which
|
|
can work, and the search that produced the URL stalls there.
|
|
"""
|
|
read = ReadFileTool(workspace=tmp_path)
|
|
|
|
for url in ("https://example.com/paper.pdf", "http://example.com/paper.pdf"):
|
|
answer = await read.execute(path=url)
|
|
assert isinstance(answer, str)
|
|
assert "web_fetch" in answer, answer
|
|
assert "File not found" not in answer, answer
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_file_url_is_answered_with_the_path_it_means(tmp_path: Path):
|
|
"""A file:// URL does name a local file, so the next step is the path, not a fetch."""
|
|
target = tmp_path / "notes.md"
|
|
target.write_text("facts", encoding="utf-8")
|
|
|
|
answer = await ReadFileTool(workspace=tmp_path).execute(path=f"file://{target}")
|
|
|
|
assert isinstance(answer, str)
|
|
assert str(target) in answer
|
|
assert "web_fetch" not in answer
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_path_that_is_merely_missing_still_says_so(tmp_path: Path):
|
|
"""The URL wording is an extra branch, not a replacement.
|
|
|
|
A relative path with a directory in it must not read as a host name.
|
|
"""
|
|
read = ReadFileTool(workspace=tmp_path)
|
|
|
|
for missing in ("ghost.md", "docs/readme.md"):
|
|
answer = await read.execute(path=missing)
|
|
assert isinstance(answer, str)
|
|
assert answer.startswith("Error: File not found:")
|
|
assert "web_fetch" not in answer
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# What an approval prompt shows before the write happens
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_a_write_over_an_existing_file_previews_as_a_diff(tmp_path: Path):
|
|
target = tmp_path / "letter.md"
|
|
target.write_text("dear friend\nold line\nbye\n", encoding="utf-8")
|
|
|
|
view = WriteFileTool(workspace=tmp_path).approval_evidence(
|
|
{"path": "letter.md", "content": "dear friend\nnew line\nbye\n"}
|
|
)
|
|
|
|
assert view["path"] == str(target.resolve())
|
|
assert view["created"] is False
|
|
assert "-old line" in view["diff"] and "+new line" in view["diff"]
|
|
assert target.read_text(encoding="utf-8") == "dear friend\nold line\nbye\n", "a preview writes nothing"
|
|
|
|
|
|
def test_a_write_to_a_new_file_previews_as_a_creation(tmp_path: Path):
|
|
view = WriteFileTool(workspace=tmp_path).approval_evidence({"path": "fresh.txt", "content": "one\ntwo\n"})
|
|
|
|
assert view["created"] is True
|
|
assert "+one" in view["diff"] and "+two" in view["diff"]
|
|
assert not (tmp_path / "fresh.txt").exists()
|
|
|
|
|
|
def test_an_append_previews_against_the_whole_file(tmp_path: Path):
|
|
target = tmp_path / "log.txt"
|
|
target.write_text("one\n", encoding="utf-8")
|
|
|
|
view = WriteFileTool(workspace=tmp_path).approval_evidence(
|
|
{"path": "log.txt", "content": "two\n", "mode": "append"}
|
|
)
|
|
|
|
assert "+two" in view["diff"] and "-one" not in view["diff"]
|
|
|
|
|
|
def test_a_file_that_cannot_be_read_as_text_previews_without_a_diff(tmp_path: Path):
|
|
target = tmp_path / "blob.bin"
|
|
target.write_bytes(b"\xff\xfe\x00binary")
|
|
|
|
view = WriteFileTool(workspace=tmp_path).approval_evidence({"path": "blob.bin", "content": "text"})
|
|
|
|
assert view == {"path": str(target.resolve()), "created": False}
|
|
|
|
|
|
def test_a_path_outside_the_fence_previews_as_the_bare_path(tmp_path: Path):
|
|
tool = WriteFileTool(workspace=tmp_path, allowed_dirs=(tmp_path,))
|
|
|
|
assert tool.approval_evidence({"path": "/etc/hosts", "content": "x"}) == {"path": "/etc/hosts"}
|
|
|
|
|
|
def test_an_edit_previews_as_the_diff_of_its_two_snippets(tmp_path: Path):
|
|
(tmp_path / "a.py").write_text("x = 1\ny = 2\n", encoding="utf-8")
|
|
|
|
view = EditFileTool(workspace=tmp_path).approval_evidence(
|
|
{"path": "a.py", "old_text": "y = 2", "new_text": "y = 3"}
|
|
)
|
|
|
|
assert view["path"] == str((tmp_path / "a.py").resolve())
|
|
assert view["created"] is False
|
|
assert "-y = 2" in view["diff"] and "+y = 3" in view["diff"]
|
|
|
|
|
|
def test_the_file_tools_declare_the_layout_a_prompt_draws_them_in(tmp_path: Path):
|
|
assert WriteFileTool(workspace=tmp_path).approval_kind == "file.write"
|
|
assert EditFileTool(workspace=tmp_path).approval_kind == "file.write"
|
|
assert ReadFileTool(workspace=tmp_path).approval_kind == ""
|
|
|
|
|
|
def test_a_file_too_large_to_preview_is_not_read_at_all(tmp_path: Path):
|
|
"""The preview runs on the gate's path before the call; a file past the cap
|
|
is answered with its path alone rather than read and diffed."""
|
|
from raven.agent.tools.filesystem import _PREVIEW_MAX_BYTES
|
|
|
|
target = tmp_path / "big.log"
|
|
target.write_bytes(b"x" * (_PREVIEW_MAX_BYTES + 1))
|
|
|
|
view = WriteFileTool(workspace=tmp_path).approval_evidence({"path": "big.log", "content": "small"})
|
|
|
|
assert view == {"path": str(target.resolve()), "created": False}
|
|
|
|
|
|
# --- a page of a text file, read a chunk at a time ------------------------------------
|
|
|
|
|
|
_EVERY_BREAK = (
|
|
b"one\n"
|
|
b"two\r\n"
|
|
b"three\r"
|
|
b"four\x0b"
|
|
b"five\x0c"
|
|
b"six\x1c"
|
|
b"seven\x1d"
|
|
b"eight\x1e"
|
|
b"nine\xc2\x85"
|
|
b"ten\xe2\x80\xa8"
|
|
b"eleven\xe2\x80\xa9"
|
|
b"\n"
|
|
b"twelve \xc3\xa9 last, with no break after it"
|
|
)
|
|
|
|
|
|
@pytest.mark.parametrize("chunk", [1, 2, 3, 4, 7, 64])
|
|
def test_a_page_is_the_lines_splitlines_gives_whatever_the_chunk_edge(monkeypatch, chunk: int):
|
|
"""Every break ``str.splitlines`` knows, each landing on a chunk edge for some chunk size."""
|
|
from io import BytesIO
|
|
|
|
from raven.agent.tools import filesystem
|
|
|
|
monkeypatch.setattr(filesystem, "_PAGE_CHUNK", chunk)
|
|
want = _EVERY_BREAK.decode("utf-8").splitlines()
|
|
assert len(want) == 13 and want[11] == "", "a blank line between two breaks counts; a trailing break does not"
|
|
|
|
for start, limit in ((0, 2000), (0, 1), (1, 3), (8, 2), (12, 1), (13, 5)):
|
|
page, total = filesystem._page_of_lines(BytesIO(_EVERY_BREAK), start, limit)
|
|
assert total == 13 and [line.decode("utf-8") for line in page] == want[start : start + limit]
|
|
assert filesystem._page_of_lines(BytesIO(b""), 0, 5) == ([], 0)
|
|
assert filesystem._page_of_lines(BytesIO(b"\n"), 0, 5) == ([b""], 1)
|
|
|
|
|
|
@pytest.mark.parametrize("chunk", [1, 3, 8, 64])
|
|
def test_a_line_longer_than_a_chunk_is_whole_on_its_page_and_only_counted_off_it(monkeypatch, chunk: int):
|
|
from io import BytesIO
|
|
|
|
from raven.agent.tools import filesystem
|
|
|
|
monkeypatch.setattr(filesystem, "_PAGE_CHUNK", chunk)
|
|
text = b"first\n" + b"L" * 50 + b"\r\nthird\n\n" + b"M" * 30
|
|
want = text.decode("utf-8").splitlines()
|
|
assert len(want) == 5
|
|
|
|
for start, limit in ((0, 1), (0, 2), (1, 1), (1, 2), (2, 5), (4, 1), (5, 1)):
|
|
page, total = filesystem._page_of_lines(BytesIO(text), start, limit)
|
|
assert total == 5 and [line.decode("utf-8") for line in page] == want[start : start + limit], (start, limit)
|
|
|
|
|
|
def test_a_long_line_is_looked_at_once_whatever_its_length(monkeypatch):
|
|
"""Each chunk is split once, never the whole of the line again: a long line costs its length, not its square.
|
|
|
|
Held whole and split again with every chunk, a file with no break in it
|
|
took half a minute at 256 MiB where the whole-file read took under a
|
|
second (measured 2026-10-13).
|
|
"""
|
|
from io import BytesIO
|
|
|
|
from raven.agent.tools import filesystem
|
|
|
|
seen: list[int] = []
|
|
split = filesystem._split_lines
|
|
monkeypatch.setattr(filesystem, "_split_lines", lambda body: (seen.append(len(body)), split(body))[1])
|
|
monkeypatch.setattr(filesystem, "_PAGE_CHUNK", 16)
|
|
one_line = b"x" * 1000 + b"\xe2\x80" + b"y" * 500 # a held-back pair inside it, and no break at all
|
|
|
|
page, total = filesystem._page_of_lines(BytesIO(one_line), 0, 5)
|
|
assert (page, total) == ([one_line], 1)
|
|
assert max(seen) <= 16 + 2 and sum(seen) == len(one_line), "every byte looked at once, plus a held-back pair"
|
|
|
|
seen.clear()
|
|
assert filesystem._page_of_lines(BytesIO(one_line + b"\nnext"), 1, 5) == ([b"next"], 2)
|
|
assert max(seen) <= 16 + 2, "the line that is not on the page is counted, not kept"
|
|
|
|
# A byte held back at the very end: a break that ends the last line, or part of it.
|
|
assert filesystem._page_of_lines(BytesIO(b"abc\r"), 0, 5) == ([b"abc"], 1)
|
|
assert filesystem._page_of_lines(BytesIO(b"abc\xe2"), 0, 5) == ([b"abc\xe2"], 1)
|
|
assert filesystem._page_of_lines(BytesIO(b"\r"), 0, 5) == ([b""], 1)
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_read_file_pages_a_file_without_holding_all_of_it(tmp_path: Path, monkeypatch):
|
|
from raven.agent.tools import filesystem
|
|
|
|
target = tmp_path / "breaks.txt"
|
|
target.write_bytes(_EVERY_BREAK)
|
|
monkeypatch.setattr(filesystem, "_PAGE_CHUNK", 5)
|
|
read = ReadFileTool(workspace=tmp_path)
|
|
|
|
first = await read.execute(path=str(target), limit=2)
|
|
assert first == "1| one\n2| two\n\n(Showing lines 1-2 of 13. Use offset=3 to continue.)"
|
|
last = await read.execute(path=str(target), offset=12)
|
|
assert last == "12| \n13| twelve \u00e9 last, with no break after it\n\n(End of file \u2014 13 lines total)"
|
|
assert await read.execute(path=str(target), offset=14) == "Error: offset 14 is beyond end of file (13 lines)"
|
|
|
|
# A byte that is not UTF-8 refuses the page it is on, as the whole-file decode refused the file.
|
|
target.write_bytes(b"fine\n\xff broken\n")
|
|
assert (
|
|
await read.execute(path=str(target), limit=1)
|
|
) == "1| fine\n\n(Showing lines 1-1 of 2. Use offset=2 to continue.)"
|
|
assert str(await read.execute(path=str(target), offset=2)).startswith("Error reading file: 'utf-8' codec")
|
|
|
|
|
|
# --- a line longer than a page --------------------------------------------------------------
|
|
|
|
|
|
_NOTE = (
|
|
"(Line {n} is {length} characters long; the first {room} are shown. offset moves by lines and cannot reach "
|
|
"the rest of it: grep tells whether a piece is in it, and a shell command that cuts by bytes, such as "
|
|
"cut -c, shows another stretch.)"
|
|
)
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_line_longer_than_a_page_shows_its_head_and_says_how_to_see_the_rest(tmp_path: Path):
|
|
"""One line over the page's character budget: the page used to be empty, with offset=1 to continue.
|
|
|
|
A minified script, a one-line JSON, a CSV row of a few hundred thousand
|
|
characters: the trim kept whole lines within the budget, the first line
|
|
did not fit, so nothing was shown and the footer sent the reader back to
|
|
the line it had just not shown (measured 2026-10-15: "(Showing lines 1-0
|
|
of 1. Use offset=1 to continue.)").
|
|
"""
|
|
target = tmp_path / "bundle.min.js"
|
|
target.write_text("x" * 200_000, encoding="utf-8")
|
|
read = ReadFileTool(workspace=tmp_path)
|
|
|
|
got = await read.execute(path=str(target))
|
|
|
|
room = ReadFileTool._MAX_CHARS - len("1| ")
|
|
body, note, footer = got.split("\n\n")
|
|
assert body == "1| " + "x" * room
|
|
assert note == _NOTE.format(n=1, length=200_000, room=room)
|
|
assert footer == "(End of file — 1 lines total)"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_paging_moves_past_a_line_longer_than_a_page_instead_of_going_round(tmp_path: Path):
|
|
"""The lines before it fill a page of their own; the page after shows its head and moves on to the next line."""
|
|
target = tmp_path / "rows.csv"
|
|
target.write_text("header\n" + "y" * 200_000 + "\nlast\n", encoding="utf-8")
|
|
read = ReadFileTool(workspace=tmp_path)
|
|
|
|
first = await read.execute(path=str(target))
|
|
second = await read.execute(path=str(target), offset=2)
|
|
third = await read.execute(path=str(target), offset=3)
|
|
|
|
assert first == "1| header\n\n(Showing lines 1-1 of 3. Use offset=2 to continue.)"
|
|
room = ReadFileTool._MAX_CHARS - len("2| ")
|
|
assert second == (
|
|
"2| "
|
|
+ "y" * room
|
|
+ "\n\n"
|
|
+ _NOTE.format(n=2, length=200_000, room=room)
|
|
+ "\n\n(Showing lines 2-2 of 3. Use offset=3 to continue.)"
|
|
)
|
|
assert third == "3| last\n\n(End of file — 3 lines total)"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_tool_s_own_cut_per_line_lands_before_the_page_s_budget(tmp_path: Path):
|
|
"""A tool with a budget per line (Raven-Code's read: 2000, with a mark) cuts on the hook, so minified lines share a page."""
|
|
|
|
class Cut(ReadFileTool):
|
|
def _cut_line(self, line: str) -> str:
|
|
return line[:8] + "..." if len(line) > 8 else line
|
|
|
|
target = tmp_path / "bundle.min.js"
|
|
target.write_text("\n".join("y" * 200_000 for _ in range(3)) + "\nend\n", encoding="utf-8")
|
|
|
|
got = await Cut(workspace=tmp_path).execute(path=str(target))
|
|
|
|
assert got == "1| yyyyyyyy...\n2| yyyyyyyy...\n3| yyyyyyyy...\n4| end\n\n(End of file \u2014 4 lines total)"
|
|
assert "(Line 1 is" not in got and "(Showing lines" not in got
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_first_line_that_just_fills_the_page_is_shown_whole_and_the_page_moves_on(tmp_path: Path):
|
|
"""At the edge the budget counts the line break too: the line fits, and is not cut or noted."""
|
|
room = ReadFileTool._MAX_CHARS - len("1| ")
|
|
target = tmp_path / "edge.txt"
|
|
target.write_text("z" * room + "\nnext\n", encoding="utf-8")
|
|
read = ReadFileTool(workspace=tmp_path)
|
|
|
|
got = await read.execute(path=str(target))
|
|
|
|
assert got == "1| " + "z" * room + "\n\n(Showing lines 1-1 of 2. Use offset=2 to continue.)"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# The Windows reserved-name fence
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"segment",
|
|
[
|
|
"nul",
|
|
"NUL",
|
|
"nul.md",
|
|
"Nul.Md",
|
|
"nul.tar.gz",
|
|
"con",
|
|
"con.txt",
|
|
"aux",
|
|
"prn",
|
|
"com1",
|
|
"com9",
|
|
"lpt1",
|
|
"lpt9",
|
|
"com¹",
|
|
"COM¹.txt",
|
|
"LPT²",
|
|
"lpt³.md",
|
|
"com1.md",
|
|
"conin$",
|
|
"conout$",
|
|
"NUL:",
|
|
"nul:stream",
|
|
"notes.md:",
|
|
"nul ",
|
|
"nul.",
|
|
"nul .md",
|
|
"notes.md.",
|
|
"notes.md ",
|
|
"...",
|
|
],
|
|
)
|
|
def test_a_reserved_segment_is_one_in_any_spelling(segment):
|
|
assert _is_reserved_windows_component(segment)
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"segment",
|
|
[
|
|
"report.md",
|
|
"null",
|
|
"nulx",
|
|
"nulx.md",
|
|
"com0",
|
|
"lpt0",
|
|
"com0.txt",
|
|
"lpt0.txt",
|
|
"com10",
|
|
"com19",
|
|
"COM⁴",
|
|
"config.json",
|
|
"auxiliary.md",
|
|
"con an.md",
|
|
"",
|
|
".",
|
|
"..",
|
|
],
|
|
)
|
|
def test_an_ordinary_segment_is_not_reserved(segment):
|
|
assert not _is_reserved_windows_component(segment)
|
|
|
|
|
|
def test_a_reserved_name_is_refused_the_way_an_out_of_fence_path_is(monkeypatch, tmp_path: Path):
|
|
"""On Windows ``nul`` names the NUL device in every directory (CPython 95486).
|
|
|
|
``Path.resolve()`` cannot see it coming: ``ntpath.realpath`` maps only a
|
|
path that is exactly ``nul`` to ``\\\\.\\NUL`` and hands
|
|
``<workspace>\\nul`` back unchanged, so the allowed-dir check accepts it
|
|
and a write answers success for content no file received. The fence has to
|
|
catch the name itself, in any component, because Windows applies its
|
|
naming rules to directories too, and in the other shapes that lose bytes
|
|
the same way: a trailing dot or space lands them in the trimmed name while
|
|
the receipt reports the given one, and a colon lands them in an NTFS
|
|
stream no directory listing shows.
|
|
"""
|
|
monkeypatch.setattr(filesystem, "_WINDOWS", True)
|
|
for name in (
|
|
"nul",
|
|
"nul.md",
|
|
"con.md",
|
|
"subdir/nul",
|
|
"nul/report.md",
|
|
"subdir/nul.md/report.md",
|
|
"COM¹.txt",
|
|
"LPT²/log.txt",
|
|
"notes.md.",
|
|
"notes.md ",
|
|
"notes.md:hidden",
|
|
"NUL:",
|
|
):
|
|
with pytest.raises(PermissionError, match="reserved name on Windows"):
|
|
resolve_path(name, workspace=tmp_path, allowed_dirs=(tmp_path,))
|
|
|
|
|
|
def test_a_reserved_name_is_refused_when_the_path_is_absolute(monkeypatch, tmp_path: Path):
|
|
monkeypatch.setattr(filesystem, "_WINDOWS", True)
|
|
|
|
with pytest.raises(PermissionError, match="reserved name on Windows"):
|
|
resolve_path(str(tmp_path / "subdir" / "nul"), workspace=tmp_path, allowed_dirs=(tmp_path,))
|
|
|
|
|
|
def test_a_drive_letter_written_with_double_slashes_stays_in_the_reserved_check(monkeypatch, tmp_path: Path):
|
|
"""``C://work/nul`` is a drive path on Windows, not a URL.
|
|
|
|
``_URL_SCHEME_RE`` accepts a one-letter scheme, so a bare scheme test
|
|
reads ``C`` as one and skips the fence -- yet Windows parses the same
|
|
string as ``C:\\work\\nul``. The exemption only holds for schemes Windows
|
|
cannot read as a drive, i.e. two letters or more, so every reserved shape
|
|
behind a ``C://`` prefix is refused like its plain spelling.
|
|
"""
|
|
monkeypatch.setattr(filesystem, "_WINDOWS", True)
|
|
|
|
for name in ("C://work/nul", "C://work/notes.md.", "C://work/notes.md:hidden", "C://work/COM¹.txt"):
|
|
with pytest.raises(PermissionError, match="reserved name on Windows"):
|
|
resolve_path(name, workspace=tmp_path, allowed_dirs=(tmp_path,))
|
|
|
|
|
|
def test_a_multi_letter_url_scheme_is_still_left_to_the_url_hint(monkeypatch, tmp_path: Path):
|
|
"""The two-letter floor keeps the real URL exemption intact."""
|
|
monkeypatch.setattr(filesystem, "_WINDOWS", True)
|
|
|
|
resolved = resolve_path("https://example.com/nul.md", workspace=tmp_path, allowed_dirs=(tmp_path,))
|
|
|
|
assert resolved == tmp_path / "https:" / "example.com" / "nul.md"
|
|
|
|
|
|
def test_the_refusal_shows_the_path_with_repr(monkeypatch, tmp_path: Path):
|
|
"""The character at fault is often invisible, so the message quotes the path.
|
|
|
|
A bare ``Path notes.md is a reserved name`` hides the trailing space a
|
|
reader has to drop; ``!r`` makes it retryable.
|
|
"""
|
|
monkeypatch.setattr(filesystem, "_WINDOWS", True)
|
|
|
|
with pytest.raises(PermissionError) as exc_info:
|
|
resolve_path("notes.md ", workspace=tmp_path, allowed_dirs=(tmp_path,))
|
|
|
|
assert "'notes.md '" in str(exc_info.value)
|
|
|
|
|
|
def test_an_ordinary_name_resolves_with_the_gate_on(monkeypatch, tmp_path: Path):
|
|
monkeypatch.setattr(filesystem, "_WINDOWS", True)
|
|
|
|
assert resolve_path("report.md", workspace=tmp_path, allowed_dirs=(tmp_path,)) == tmp_path / "report.md"
|
|
|
|
|
|
def test_a_zero_numbered_com_name_is_an_ordinary_file_with_the_gate_on(monkeypatch, tmp_path: Path):
|
|
"""Microsoft's naming rules reserve COM/LPT 1-9; 0 never named a device."""
|
|
monkeypatch.setattr(filesystem, "_WINDOWS", True)
|
|
|
|
for name in ("com0.txt", "lpt0.txt"):
|
|
assert resolve_path(name, workspace=tmp_path, allowed_dirs=(tmp_path,)) == tmp_path / name
|
|
|
|
|
|
def test_a_url_argument_is_left_to_the_tools_url_hint_with_the_gate_on(monkeypatch, tmp_path: Path):
|
|
"""The URL's scheme colon is one of the reserved shapes; the hint must win.
|
|
|
|
``_missing`` answers a URL with the tool that can serve it, so the fence
|
|
must not refuse a URL argument first -- a scheme of two letters or more
|
|
names a resource no local drive can spell.
|
|
"""
|
|
monkeypatch.setattr(filesystem, "_WINDOWS", True)
|
|
|
|
resolved = resolve_path("https://example.com/a.md", workspace=tmp_path, allowed_dirs=(tmp_path,))
|
|
|
|
assert resolved == tmp_path / "https:" / "example.com" / "a.md"
|
|
|
|
|
|
def test_a_device_like_name_stays_legal_off_windows(monkeypatch, tmp_path: Path):
|
|
"""POSIX puts no devices in names: a workspace file called ``nul`` is a file."""
|
|
monkeypatch.setattr(filesystem, "_WINDOWS", False)
|
|
|
|
assert resolve_path("nul", workspace=tmp_path, allowed_dirs=(tmp_path,)) == tmp_path / "nul"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_write_to_a_device_name_answers_an_error_and_writes_nothing(monkeypatch, tmp_path: Path):
|
|
"""The defect was a success receipt for content no file received."""
|
|
monkeypatch.setattr(filesystem, "_WINDOWS", True)
|
|
|
|
answer = await WriteFileTool(workspace=tmp_path).execute(path="nul", content="hello\n")
|
|
|
|
assert isinstance(answer, str)
|
|
assert "reserved name on Windows" in answer
|
|
# On Windows the device itself answers ``exists()``, so the directory
|
|
# entries are what can prove no file was created.
|
|
assert list(tmp_path.iterdir()) == []
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_write_to_a_colon_device_spelling_answers_an_error_and_writes_nothing(monkeypatch, tmp_path: Path):
|
|
"""``NUL:`` is the device itself; its receipt must not claim a save either."""
|
|
monkeypatch.setattr(filesystem, "_WINDOWS", True)
|
|
|
|
answer = await WriteFileTool(workspace=tmp_path).execute(path="NUL:", content="hello\n")
|
|
|
|
assert isinstance(answer, str)
|
|
assert "reserved name on Windows" in answer
|
|
assert list(tmp_path.iterdir()) == []
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_write_to_a_trailing_dot_name_answers_an_error_and_writes_nothing(monkeypatch, tmp_path: Path):
|
|
"""Windows trims the dot before creating: the bytes land in ``notes.md``."""
|
|
monkeypatch.setattr(filesystem, "_WINDOWS", True)
|
|
|
|
answer = await WriteFileTool(workspace=tmp_path).execute(path="notes.md.", content="hello\n")
|
|
|
|
assert isinstance(answer, str)
|
|
assert "reserved name on Windows" in answer
|
|
assert list(tmp_path.iterdir()) == []
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_write_to_a_zero_numbered_com_name_writes_a_real_file(monkeypatch, tmp_path: Path):
|
|
"""The control side of the 1-9 rule: ``com0.txt`` is a file Windows creates."""
|
|
monkeypatch.setattr(filesystem, "_WINDOWS", True)
|
|
|
|
answer = await WriteFileTool(workspace=tmp_path).execute(path="com0.txt", content="hello\n")
|
|
|
|
assert isinstance(answer, ToolResult)
|
|
assert answer.model_text.startswith("Successfully wrote ")
|
|
assert [p.name for p in tmp_path.iterdir()] == ["com0.txt"]
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_read_of_a_device_name_answers_an_error_not_a_missing_file(monkeypatch, tmp_path: Path):
|
|
""" "File not found" reads as a misspelling; the name is the problem itself."""
|
|
monkeypatch.setattr(filesystem, "_WINDOWS", True)
|
|
|
|
answer = await ReadFileTool(workspace=tmp_path).execute(path="nul.md")
|
|
|
|
assert isinstance(answer, str)
|
|
assert "reserved name on Windows" in answer
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_read_of_a_url_still_answers_the_fetch_hint_with_the_gate_on(monkeypatch, tmp_path: Path):
|
|
"""The gate must not swallow the URL answer: its colon is not a stream."""
|
|
monkeypatch.setattr(filesystem, "_WINDOWS", True)
|
|
|
|
answer = await ReadFileTool(workspace=tmp_path).execute(path="https://example.com/a.md")
|
|
|
|
assert isinstance(answer, str)
|
|
assert "web_fetch" in answer
|
|
|
|
|
|
REMOTE_RESERVED_SPELLINGS = (
|
|
"box:raven-work/t1/bundle.js",
|
|
"root@203.0.113.7:/srv/x.md",
|
|
"box:raven-work/nul",
|
|
"box:nul",
|
|
"root@203.0.113.7:/srv/aux",
|
|
"box:raven-work/com1",
|
|
"box:raven-work/nul.md",
|
|
"box:raven-work/nul:hidden",
|
|
"scp://box:2222//raven-work/nul",
|
|
)
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
@pytest.mark.parametrize("spelling", REMOTE_RESERVED_SPELLINGS)
|
|
@pytest.mark.parametrize(
|
|
"tool_name,call",
|
|
[
|
|
("read_file", {}),
|
|
("write_file", {"content": "hello\n"}),
|
|
("edit_file", {"old_text": "a", "new_text": "b"}),
|
|
("list_dir", {}),
|
|
],
|
|
)
|
|
async def test_a_remote_path_reaches_its_machine_with_the_gate_on(
|
|
monkeypatch, tmp_path: Path, spelling: str, tool_name: str, call: dict
|
|
):
|
|
"""A remote path's components are that machine's files, ordinary names there.
|
|
|
|
``nul``, ``aux`` and ``com1`` are legal file names on a Linux machine, so a
|
|
remote spelling must reach ``remote_files.answer`` whatever its components
|
|
say. Only the head, before the separator, could be the device instead of a
|
|
machine. The native Windows CI job caught the earlier guard refusing
|
|
``box:raven-work/...``; the remote reserved-name shapes are the same class
|
|
one component further in.
|
|
"""
|
|
monkeypatch.setattr(filesystem, "_WINDOWS", True)
|
|
seen: list[tuple[str, str]] = []
|
|
|
|
async def answer(tool: str, path: str, params: dict) -> str | None:
|
|
if not remote_path.looks_remote(path):
|
|
return None
|
|
seen.append((tool, path))
|
|
return f"Error: {path} is on another machine, and this process does not read registered machines' files"
|
|
|
|
monkeypatch.setattr(remote_files, "answer", answer)
|
|
tools = {
|
|
"read_file": ReadFileTool,
|
|
"write_file": WriteFileTool,
|
|
"edit_file": EditFileTool,
|
|
"list_dir": ListDirTool,
|
|
}
|
|
|
|
out = await tools[tool_name](workspace=tmp_path).execute(path=spelling, **call)
|
|
|
|
assert isinstance(out, str)
|
|
assert "reserved name on Windows" not in out
|
|
assert "another machine" in out
|
|
assert seen == [(tool_name, spelling)]
|
|
assert list(tmp_path.iterdir()) == []
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_local_reserved_component_still_answers_the_refusal_with_the_gate_on(monkeypatch, tmp_path: Path):
|
|
"""The control for the test above: a local spelling keeps every component checked."""
|
|
monkeypatch.setattr(filesystem, "_WINDOWS", True)
|
|
seen: list[tuple[str, str]] = []
|
|
|
|
async def answer(tool: str, path: str, params: dict) -> str | None:
|
|
if not remote_path.looks_remote(path):
|
|
return None
|
|
seen.append((tool, path))
|
|
return f"Error: {path} is on another machine"
|
|
|
|
monkeypatch.setattr(remote_files, "answer", answer)
|
|
|
|
for spelling in ("subdir/nul", "nul/report.md", "notes.md.", "notes.md "):
|
|
out = await WriteFileTool(workspace=tmp_path).execute(path=spelling, content="hello\n")
|
|
assert isinstance(out, str)
|
|
assert "reserved name on Windows" in out, spelling
|
|
assert seen == []
|
|
assert list(tmp_path.iterdir()) == []
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_colon_device_spelling_never_reaches_its_machine_with_the_gate_on(monkeypatch, tmp_path: Path):
|
|
"""The control for the test above: ``NUL:`` is the device, not a host named ``NUL``."""
|
|
monkeypatch.setattr(filesystem, "_WINDOWS", True)
|
|
seen: list[str] = []
|
|
|
|
async def answer(tool: str, path: str, params: dict) -> str:
|
|
seen.append(path)
|
|
return f"Error: {path} is on another machine"
|
|
|
|
monkeypatch.setattr(remote_files, "answer", answer)
|
|
|
|
refused = await WriteFileTool(workspace=tmp_path).execute(path="NUL:", content="hello\n")
|
|
|
|
assert isinstance(refused, str)
|
|
assert "reserved name on Windows" in refused
|
|
assert seen == []
|
|
assert list(tmp_path.iterdir()) == []
|